URL: https://testingbot.com/ # Cross Browser Testing on 6100+ Real Browsers & Devices Run automated, manual, visual and AI-assisted tests on 6100+ real browsers and physical iOS and Android devices, with parallel execution and CI/CD integrations. Trusted by some of the world's most innovative companies [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) No credit card required 1+bn Tests Executed 180k Active Users 6100+ Browsers & Devices About TestingBot ## All-in-one cross-browser, mobile and AI testing platform TestingBot runs your Selenium, Appium, Playwright, Cypress, Puppeteer, Espresso, XCUITest and Maestro tests on 6100+ real browsers and physical iOS and Android devices. Automated, manual, visual and AI-assisted testing with parallel execution and CI/CD integrations. ### Web Testing Test on 6100+ real browsers across Windows, macOS and Linux. #### [Live / Manual](https://testingbot.com/features/manual-browser-testing) Interactive browser testing #### [Automated](https://testingbot.com/features) Selenium, Playwright, Puppeteer & Cypress #### [AI Testing](https://testingbot.com/features/ai-testing) AI-driven testing & MCP integration #### [Visual Testing](https://testingbot.com/features/screenshot-testing) Screenshot comparison of websites ### Mobile App Testing Test on real physical iOS and Android devices. #### [Live / Manual](https://testingbot.com/features/manual-mobile-testing) Interactive device testing #### [Automated](https://testingbot.com/real-device-testing) Appium, Espresso, XCUITest & Maestro #### [AI Testing](https://testingbot.com/features/ai-testing) AI-driven testing on real devices #### [Visual Testing](https://testingbot.com/features/screenshot-testing) App screenshot comparison ![App screenshot comparison](https://testingbot.com/assets/features/manual-flow-b2c093218b8db3363e5903a6197ce47efa4361cd8c114738489bbc673b5d168a.jpg) ## Cross-Browser Testing Made Simple Get one-click access to real browsers running on macOS, Windows and Linux platforms. Switch platforms, locations and screen resolutions instantly. Test across all versions of Internet Explorer (IE8 to IE11), Edge, Chrome, Safari, Firefox and Opera. ![Chrome](https://testingbot.com/assets/home/chrome-97eb1fabe899086fad592ef3ed29d82d0a206a2002d5c3be62079a3163391e1d.svg) Chrome ![Safari](https://testingbot.com/assets/environments/svg/safari-e4837bb5f9a9729fef843af4180372e975c62587808b8ab479f615f3e240e858.svg) Safari ![Firefox](https://testingbot.com/assets/home/firefox-a8ee4666d002663e9868f9b15eb50f681cc03839ae1ead6432353c617496b95d.svg) Firefox ![Edge](https://testingbot.com/assets/home/edge-8204f73ccd649193b2a4d066840b4431b982015625e0c42a95bf91b5eec127b2.svg) Edge [Start Testing](https://testingbot.com/features/manual-browser-testing) ## Why teams choose TestingBot ### Improve test coverage Cross-browser, real device and visual testing on 6100+ browser and device combinations. ### Boost productivity with AI AI-powered test creation and execution. Write tests in natural language, let AI do the rest. ### Accelerate testing cycles Run tests in parallel across multiple browsers and devices. Ship faster with confidence. ## Works with your favorite testing frameworks Native support for all major web and mobile testing frameworks. Get started in minutes. ### Web Testing [Selenium](https://testingbot.com/features/automation/selenium) [Playwright](https://testingbot.com/features/automation/playwright) [Cypress](https://testingbot.com/features/automation/cypress) [Puppeteer](https://testingbot.com/features/automation/puppeteer) [WebdriverIO](https://testingbot.com/support/web-automate/selenium/nodejs/webdriverio) [RobotFramework](https://testingbot.com/support/web-automate/selenium/robotframework) ### Mobile Testing [Appium](https://testingbot.com/features/automation/appium) [Espresso](https://testingbot.com/features/automation/espresso) [XCUITest](https://testingbot.com/features/automation/xcuitest) [Maestro](https://testingbot.com/features/automation/maestro) [Flutter](https://testingbot.com/support/app-automate/appium/flutter) [React Native](https://testingbot.com/features/automation/appium) ### Languages & SDKs [Java](https://testingbot.com/support/web-automate/selenium/java) [Python](https://testingbot.com/support/web-automate/selenium/python) [NodeJS](https://testingbot.com/support/web-automate/selenium/nodejs) [C# / .NET](https://testingbot.com/support/web-automate/selenium/csharp) [Ruby](https://testingbot.com/support/web-automate/selenium/ruby) [PHP](https://testingbot.com/support/web-automate/selenium/php) [View all documentation](https://testingbot.com/support) Real Device Cloud ## Test on real iOS & Android devices Access hundreds of real physical devices for manual and automated testing. iPhones, iPads, Samsung Galaxy, Pixel, Smart TVs and more. [Appium](https://testingbot.com/features/automation/appium) [Espresso](https://testingbot.com/features/automation/espresso) [XCUITest](https://testingbot.com/features/automation/xcuitest) [Maestro](https://testingbot.com/features/automation/maestro) [Explore device cloud](https://testingbot.com/real-device-testing) ![Real device testing](https://testingbot.com/assets/hero/app-testing-eb9e2816aa7dce01b215db1df0a54c361c2ff9b8c30850a5dd00f088c9255cb3.webp) AI-Powered Testing ## AI Test Agent Create and run tests without writing code. Simply describe what you want to test in plain language. TestingBot AI understands your application and generates reliable test steps automatically. [Try AI Testing](https://testingbot.com/features/ai-testing) ![AI Testing Interface](https://testingbot.com/assets/home/ai-chat-24cc1ab8234690bad4a866888a2bd8e11f1301b7e21ad86be5d4d61be838707a.webp) ![Accessibility Testing Dashboard](https://testingbot.com/assets/features/accessibility/accessibility_ui-b64827a2253e7d403626e287a9da43bbf597d15b8e61fce56a53d8f3ffb9db1a.jpg) Accessibility Testing ## WCAG Compliance Testing Identify and fix accessibility issues across your websites. Ensure WCAG compliance and deliver an inclusive experience for all users. Meet legal requirements like the European Accessibility Act (EAA), reach a broader audience and improve SEO with accessibility best practices. WCAG 2.1 Compliance Scheduled Testing Detailed Reports Severity Classification [Explore Accessibility Testing](https://testingbot.com/features/accessibility-testing) ## Trusted by teams worldwide Join thousands of companies shipping better software with TestingBot 1B+ Tests Executed 180,000+ Active Users 6,100+ Browsers & Devices 106 Countries GDPR Compliant SAML SSO Single Sign-On Star Level 1 Certified 99.9% Uptime SLA ## Seamlessly integrate into your workflow Connect TestingBot with your CI/CD pipeline, issue trackers and communication tools. [Jenkins](https://testingbot.com/support/integrations/ci-cd/jenkins) [GitHub](https://testingbot.com/support/integrations/ci-cd/github-actions) [GitLab](https://testingbot.com/support/integrations/ci-cd/gitlab) [CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci) [Travis CI](https://testingbot.com/support/integrations/ci-cd/travis-ci) [Slack](https://testingbot.com/support/integrations/slack) [Jira](https://testingbot.com/support/integrations/jira) [Azure](https://testingbot.com/support/integrations/ci-cd/azure) [Bitbucket](https://testingbot.com/support/integrations/ci-cd/bitbucket) [TeamCity](https://testingbot.com/support/integrations/ci-cd/teamcity) [Bamboo](https://testingbot.com/support/integrations/ci-cd/bamboo) [App Center](https://testingbot.com/support/integrations/appcenter) [View all integrations](https://testingbot.com/integrations) FAQ ## Frequently Asked Questions What is cross-browser testing? Cross-browser testing verifies that a website or web application works correctly across different browsers (Chrome, Safari, Firefox, Edge, Internet Explorer), browser versions, operating systems and screen sizes. It catches rendering inconsistencies, JavaScript compatibility issues and CSS bugs before users encounter them. TestingBot runs your tests on 6100+ real browser and OS combinations in parallel. How do I test a website on multiple browsers? Use a cross-browser testing platform like TestingBot. Sign up, point your Selenium, Playwright, Cypress or Puppeteer test scripts at the TestingBot grid by changing the WebDriver URL, and your tests run in parallel across thousands of real browsers and devices. You can also start live, interactive sessions on any browser without writing code. Which test automation frameworks does TestingBot support? TestingBot natively supports Selenium, Appium, Playwright, Cypress, Puppeteer, Espresso, XCUITest and Maestro. SDKs are available for Java, JavaScript, TypeScript, Python, Ruby, C#, PHP and Go. AI assistants like Claude Code and Cursor can also access the TestingBot REST API via a [Model Context Protocol (MCP) server](https://testingbot.com/support/ai/mcp) to inspect test runs and debug failures from your editor. Does TestingBot support real iOS and Android devices? Yes. TestingBot operates a cloud of physical iOS and Android devices, including iPhones (from iPhone 6s to iPhone 17 Pro), iPads, Samsung Galaxy phones, Pixel devices and tablets. You can run automated Appium, Espresso, XCUITest or Maestro tests on real hardware, or take manual control via the [live testing interface](https://testingbot.com/features/manual-mobile-testing). How is TestingBot different from BrowserStack, Sauce Labs and LambdaTest? TestingBot is more affordable than BrowserStack and Sauce Labs (typically 30 to 40 percent cheaper), includes real device testing in every paid plan, hosts data in the European Union for GDPR compliance, and ships features faster, Apple Silicon support arrived more than ten months before Sauce Labs. See the dedicated comparisons: [BrowserStack alternative](https://testingbot.com/browserstack-alternative), [Sauce Labs alternative](https://testingbot.com/saucelabs-alternative), [LambdaTest alternative](https://testingbot.com/lambdatest-alternative). Does TestingBot support AI-driven testing? Yes. TestingBot offers [codeless AI testing](https://testingbot.com/support/ai/codeless), describe in plain language what you want to test, and an AI agent creates and runs the test on the browsers you choose, reporting back any failures. TestingBot also exposes its REST API via a [Model Context Protocol (MCP) server](https://testingbot.com/support/ai/mcp) so AI assistants like Claude Code and Cursor can inspect test runs, fetch logs and debug failures from your editor. See the full [AI testing overview](https://testingbot.com/support/ai) for integrations with Magnitude, Stagehand and Notte. Where is TestingBot data hosted? TestingBot is built and hosted in Europe. All test data, video recordings and logs are stored on European infrastructure (EU and Switzerland), making TestingBot a strong choice for GDPR-compliant testing programs and for teams with European data-residency requirements. Is there a free trial? Yes. TestingBot offers a free plan and a free trial of paid features. The free plan needs no credit card: it runs automated tests, live manual sessions, screenshot tests and visual comparisons, including Safari on macOS, on a recent version of each browser with a limited number of live macOS sessions. The 7-day full-access trial asks for a card, charges nothing during the trial, and unlocks every browser version, every macOS release and real iOS and Android devices. No credit card required ## Ready to get started? Join thousands of teams shipping better software with TestingBot [Start Free Trial](https://testingbot.com/users/sign_up) [View pricing](https://testingbot.com/pricing) Free 14-day trial 5-minute setup 24/7 support --- URL: https://testingbot.com/features # All-in-One Cross Browser & Mobile Testing Platform TestingBot is a cloud testing platform that runs your Selenium, Appium, Playwright, Cypress, Puppeteer, Espresso, XCUITest and Maestro tests on 6100+ real desktop browsers and physical iOS and Android devices. Test in parallel, debug with video and screenshot recording and integrate with any CI/CD system. [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) - **Browsers & devices:** 6100+ - **Uptime SLA:** 99.9% ## Trusted by some of the world's most innovative companies ## All-in-one cross browser testing platform The TestingBot cloud brings you the power to test your website and mobile app on [+6100 browsers & devices](https://testingbot.com/support/web-automate/browsers). Test on the latest Chrome, Firefox, Edge and Safari, plus legacy versions like Internet Explorer 11, all running on real Windows, macOS and Linux machines. Let us handle your testing infrastructure while you focus on development. Run tests effortlessly on our maintained cloud of desktop and mobile platforms. [Start free](https://testingbot.com/users/sign_up) Built for scale ## 6100+ real desktop and mobile browser combinations Test on the widest range of desktop and mobile devices in our real device cloud. We handle setup, procurement and maintenance, so you can focus entirely on testing. ### Windows, macOS & Linux One-click access to every macOS, Linux and Windows version. From Yosemite to macOS Tahoe, and Windows 7 to Windows 11. We've got you covered. ### Every browser, all versions Edge, Safari, Firefox, Chrome, Opera and IE, constantly updated and instantly available. ### Security Single-Use VMs and real devices, reset after each use. ### Real Android and iOS devices Test across real iPhone, iPads, Google, OnePlus, Xiaomi and Samsung phones and tabs. ### No setup and maintenance Instantly access 6100+ devices and browsers. Experience zero setup or maintenance hassles, and focus only on testing. ### Performance Our infrastructure is built for performance to eliminate test flakiness and latency. Automation frameworks ## Test Automation Frameworks TestingBot supports these popular Test Automation Frameworks: - [![Selenium Testing](https://testingbot.com/assets/support/selenium-266ecc0599068fc6eca0bcd9174fb46053c95cebe1c66f7a50c480f3f1a52929.svg)](https://testingbot.com/features/automation/selenium) ### [Selenium](https://testingbot.com/features/automation/selenium) Popular Browser Automation Framework, using WebDriver, which supports all major browsers. Best for: enterprise-grade browser automation across every major language. [Try with Selenium](https://testingbot.com/features/automation/selenium) - [![Appium Testing](https://testingbot.com/assets/support/appium-4b35a8bb56bae236bd84c539cee9b8bbb489a55ddf48e32d059c86685d230354.svg)](https://testingbot.com/features/automation/appium) ### [Appium](https://testingbot.com/features/automation/appium) Mobile Automation Framework, using WebDriver, runs on physical devices and simulators/emulators. Best for: native mobile app automation on real iOS & Android devices. [Try with Appium](https://testingbot.com/features/automation/appium) - [![Cypress Testing](https://testingbot.com/assets/support/cypress-409c9fd3f79115ed7676aa1df4f6a217dc4402b3366a61984924aec7b4fefa97.svg)](https://testingbot.com/features/automation/cypress) ### [Cypress](https://testingbot.com/features/automation/cypress) Javascript based E2E testing on Chrome and Firefox. Best for: fast JavaScript-only end-to-end testing. [Try with Cypress](https://testingbot.com/features/automation/cypress) - [![Puppeteer Testing](https://testingbot.com/assets/support/puppeteer-0a8cdd38af69dabd46638afbb3bffc8a372902949aa599663ec379f3e875263c.svg)](https://testingbot.com/features/automation/puppeteer) ### [Puppeteer](https://testingbot.com/features/automation/puppeteer) Chrome DevTools based Automation Framework by Google. Supports Chrome and Edge browsers. Best for: Chromium DevTools-based automation and scraping. [Try with Puppeteer](https://testingbot.com/features/automation/puppeteer) - [![Playwright Testing](https://testingbot.com/assets/support/playwright-2dcb4e75b55ac0d1135cfa47be6137b13f46a2047ce78b96a34f51b0e2cc0d96.svg)](https://testingbot.com/features/automation/playwright) ### [Playwright](https://testingbot.com/features/automation/playwright) Similar to Puppeteer, built by Microsoft, with support for Chrome and Edge. Best for: modern multi-browser testing in TypeScript, Python and .NET. [Try with Playwright](https://testingbot.com/features/automation/playwright) - [![Espresso Testing](https://testingbot.com/assets/integrations/espresso-a8ff30da0c385f89f4e694bb29fc4779ad132e8e3f4a777a38e608e0492ff610.svg)](https://testingbot.com/features/automation/espresso) ### [Espresso](https://testingbot.com/features/automation/espresso) Android Automation Framework, runs on physical devices and emulators. Best for: Android-only UI testing inside the app process. [Try with Espresso](https://testingbot.com/features/automation/espresso) - [![XCUITest Testing](https://testingbot.com/assets/integrations/xcuitest-4a935f2368661d89d3559b3c5af3377236029478ae76bb43253f3c3bee712096.svg)](https://testingbot.com/features/automation/xcuitest) ### [XCUITest](https://testingbot.com/features/automation/xcuitest) iOS Automation Framework, runs on physical devices and simulators. Best for: iOS-only UI testing with Apple's native framework. [Try with XCUITest](https://testingbot.com/features/automation/xcuitest) - [![Maestro Testing](https://testingbot.com/assets/integrations/maestro-9d782f30f13fb4efaf9928b704c7a787eb14cb9e5b20c650137bb65892334b8b.svg)](https://testingbot.com/features/automation/maestro) ### [Maestro](https://testingbot.com/features/automation/maestro) Maestro is a mobile UI testing framework, which supports testing on both iOS and Android. Best for: declarative mobile flows without writing code. [Try with Maestro](https://testingbot.com/features/automation/maestro) Comparing frameworks? See [Playwright vs Selenium](https://testingbot.com/playwright-vs-selenium) for a head-to-head breakdown. Whichever you use, [running tests in parallel](https://testingbot.com/features/parallel-testing) is what cuts the wall-clock time. Buy vs build ## TestingBot vs a DIY in-house grid Building and maintaining your own browser and device lab is expensive and slow. Here is how a hosted cloud compares to running the infrastructure yourself. | Capability | TestingBot | DIY in-house grid | | --- | --- | --- | | Browser & device coverage | 6100+ ready to use | You buy and maintain | | Setup time | Minutes | Weeks to months | | Maintenance burden | None, managed for you | Constant patching, OS upgrades and capacity planning | | Real iOS devices | iPhone, iPad, real hardware | Hard (Apple licensing & MDM) | | Parallel sessions | Scale to hundreds in parallel | Capped by your hardware | | EU data residency | Hosted in the EU | Build it yourself | | Video & screenshot recording | Recorded automatically on every test | Build it yourself | | CI/CD integrations | Jenkins, GitHub Actions, CircleCI, GitLab, Azure DevOps | DIY scripts and plugins | | Live debugging | Built-in | Custom VNC tooling | | Support | 24/7 included | Internal team only | Live mobile testing ## Take control of a device Interact with the remote mobile device, just as you would with a device in hand. Tap, scroll, zoom, swipe and more. Open a real device anywhere in the world and drive it from your browser. Tap, scroll, swipe, install your APK or IPA, capture network logs and reproduce a customer issue in real time. [Try Live Mobile Testing](https://testingbot.com/features/manual-mobile-testing) Developer experience ## Test in your favorite language Use the SDK of your choice. TestingBot integrates with Jest, Mocha, PyTest, NUnit, RSpec and other popular test frameworks, plus every major CI/CD system. hub.testingbot.com Copy JavaScript Python Java Ruby C# PHP ```javascript // Run a Selenium test on the TestingBot grid const wd = require('selenium-webdriver'); const driver = await new wd.Builder() .usingServer('https://hub.testingbot.com/wd/hub') .withCapabilities({ 'tb:options': { name: 'Checkout flow' }, browserName: 'chrome', browserVersion: 'latest', platformName: 'Windows 11' }) .build(); await driver.get('https://example.com'); await driver.quit(); ``` ```python # Run a Selenium test on the TestingBot grid from selenium import webdriver options = webdriver.ChromeOptions() options.set_capability('tb:options', {'name': 'Checkout flow'}) options.set_capability('browserVersion', 'latest') options.set_capability('platformName', 'Windows 11') driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options ) driver.get('https://example.com') driver.quit() ``` ```java // Run a Selenium test on the TestingBot grid DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("browserName", "chrome"); caps.setCapability("browserVersion", "latest"); caps.setCapability("platformName", "Windows 11"); WebDriver driver = new RemoteWebDriver( new URL("https://hub.testingbot.com/wd/hub"), caps ); driver.get("https://example.com"); driver.quit(); ``` ```ruby # Run a Selenium test on the TestingBot grid require 'selenium-webdriver' caps = Selenium::WebDriver::Remote::Capabilities.chrome( browserVersion: 'latest', platformName: 'Windows 11', 'tb:options' => { name: 'Checkout flow' } ) driver = Selenium::WebDriver.for(:remote, url: 'https://hub.testingbot.com/wd/hub', capabilities: caps ) driver.get 'https://example.com' driver.quit ``` ```csharp // Run a Selenium test on the TestingBot grid var options = new ChromeOptions(); options.AddAdditionalOption("browserVersion", "latest"); options.AddAdditionalOption("platformName", "Windows 11"); IWebDriver driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), options ); driver.Navigate().GoToUrl("https://example.com"); driver.Quit(); ``` ```php // Run a Selenium test on the TestingBot grid $capabilities = DesiredCapabilities::chrome(); $capabilities->setCapability('browserVersion', 'latest'); $capabilities->setCapability('platformName', 'Windows 11'); $driver = RemoteWebDriver::create( 'https://hub.testingbot.com/wd/hub', $capabilities ); $driver->get('https://example.com'); $driver->quit(); ``` Connected · [View documentation](https://testingbot.com/support/web-automate/selenium/nodejs) [View documentation](https://testingbot.com/support/web-automate/selenium/python) [View documentation](https://testingbot.com/support/web-automate/selenium/java) [View documentation](https://testingbot.com/support/web-automate/selenium/ruby) [View documentation](https://testingbot.com/support/web-automate/selenium/csharp) [View documentation](https://testingbot.com/support/web-automate/selenium/php) ![Test on real mobile and desktop devices](https://testingbot.com/assets/features/manual-flow-b2c093218b8db3363e5903a6197ce47efa4361cd8c114738489bbc673b5d168a.jpg) ## Test on real mobile and desktop devices TestingBot only runs real browsers on real Windows, macOS and Linux machines, never simulators. Tests on TestingBot match what your customers actually see, so your results are always accurate. Test on the latest Chrome, Firefox, Edge and Safari, plus legacy versions like Internet Explorer 11, all running on real Windows, macOS and Linux machines. Test across all versions of Internet Explorer (IE8 to IE11), Edge, Chrome, Safari, Firefox and Opera. [Start free](https://testingbot.com/users/sign_up) Secure tunnel ## Test Staged Websites Use the secure TestingBot Tunnel to easily test your staging and development environments across all browsers, ensuring seamless access behind firewalls or private networks. Test data and websites from behind your firewall on the TestingBot cloud. ![TestingBot Tunnel architecture diagram](https://testingbot.com/assets/features/tunnel-diagram-poster-8c25e19e6fe4f7276a46a96b31a0ad0e07bb4f5bf98dc192d2b70897cfbb1a26.png) [Read more](https://testingbot.com/support/tunnel) FAQ ## Frequently asked questions about TestingBot features What testing frameworks does TestingBot support? TestingBot natively supports **Selenium, Appium, Playwright, Cypress, Puppeteer, Espresso, XCUITest and Maestro** , so you can keep your existing test framework and point it at the TestingBot cloud by changing a single endpoint URL. Official SDKs ship for Java, JavaScript, TypeScript, Python, Ruby, C#, PHP and Go, and each returns the same session metadata, video recordings and logs regardless of language. Selenium, Playwright, Cypress and Puppeteer handle desktop browser automation, while Appium, Espresso, XCUITest and Maestro drive real iOS and Android devices. Beyond writing code, TestingBot offers codeless AI test creation, where you describe a scenario in plain language and an agent builds and runs it for you. AI assistants like Claude Code and Cursor can also drive TestingBot through a [Model Context Protocol (MCP) server](https://testingbot.com/support/ai/mcp), inspecting runs, fetching logs and debugging failures without leaving your editor. Can I run automated tests in parallel? Yes. Run automated Selenium, Appium, Playwright, Cypress, Puppeteer, Espresso, XCUITest and Maestro tests in parallel, from a handful of concurrent sessions on the free plan up to hundreds of parallel sessions on Enterprise. Parallel execution typically reduces total test runtime by 10× to 50× compared to a single-machine grid, because your suite is split across many real browsers and devices that all run at the same time. You control concurrency from your test configuration or CI pipeline by setting the number of parallel sessions, and TestingBot queues and distributes the tests for you. Each session is isolated on its own single-tenant virtual machine or freshly reset device, so parallel runs never share state or interfere with one another. Most teams use this to keep a growing regression suite inside the same CI time budget, turning an overnight run into one that finishes in minutes. Does TestingBot support real iOS and Android devices? Yes. TestingBot operates a cloud of physical iOS and Android devices, including iPhones (from iPhone 6s to the latest iPhone), iPads, Samsung Galaxy phones, Pixel devices and tablets, all real hardware rather than emulators or simulators. You can run automated Appium, Espresso, XCUITest or Maestro tests on these devices, or take manual control through the [live testing interface](https://testingbot.com/features/manual-mobile-testing) to tap, swipe, type and inspect elements in real time. Every device is wiped and reset to a clean state after each session, so no test data, credentials or installed apps carry over between runs. New iOS and Android releases are added quickly, real iOS 26 device support arrived the same week Apple released the OS. Because the devices are hosted in Europe, your mobile test data, video recordings and logs stay on European infrastructure (EU and Switzerland), which helps teams with GDPR or data-residency requirements. How do I test internal or staging applications with TestingBot? Use the [TestingBot Tunnel](https://testingbot.com/support/tunnel) to expose a staging or internal application to the TestingBot cloud over a secure connection. The tunnel is encrypted end-to-end, requires no firewall changes, and lets every browser and device in the cloud reach hostnames behind your VPN. What CI/CD integrations does TestingBot offer? TestingBot ships first-party plugins and integrations for Jenkins, GitHub Actions, GitLab CI, CircleCI, Azure DevOps, Bamboo, TeamCity and Travis CI. The platform also exposes a REST API and webhook system, so any CI/CD tool can launch tests, fetch results and react to failures. Can I do manual or live browser testing on TestingBot? Yes. [Live browser testing](https://testingbot.com/features/manual-browser-testing) and [live mobile testing](https://testingbot.com/features/manual-mobile-testing) let you remote-control any real browser, device, or simulator from inside your own browser: type, click, scroll, pinch and inspect. There is no SDK to install, and sessions launch in seconds. Is TestingBot GDPR-compliant and secure? TestingBot is built and hosted in Europe, GDPR-compliant by design and CSA STAR Level 1 registered. The infrastructure is single-tenant per test session: virtual machines and devices are wiped after every test. ### Sign up for a Free Trial Start testing your apps with TestingBot. [Start a free trial](https://testingbot.com/users/sign_up) Working on an open source project? [Get these features free for open source](https://testingbot.com/open-source). --- URL: https://testingbot.com/real-device-testing Real Device Cloud · iOS & Android # Real device testing online, on iOS and Android Real device testing means running your automated and manual tests on physical phones and tablets rather than on emulators. TestingBot provides a cloud of real iOS and Android devices, hosted in a secure EU datacenter, ready to run your tests in seconds, alongside hosted emulators and simulators for the faster checks you want on every commit. - Real iOS & Android devices - European-hosted, GDPR-compliant - [Appium](https://testingbot.com/features/automation/appium), [XCUITest](https://testingbot.com/features/automation/xcuitest), [Espresso](https://testingbot.com/features/automation/espresso), [Maestro](https://testingbot.com/features/automation/maestro), [Mobilewright](https://testingbot.com/support/app-automate/mobilewright) - Pristine, never jailbroken [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) Trusted by some of the world's most innovative companies Built for scale ## Reliable Device Farm Get instant access to a fast, reliable device infrastructure. ### 24/7 device availability Physical iOS and Android devices, available at any time of the day. ### Zero setup & Maintenance Focus on testing instead of worrying about setup and maintenance of a mobile device farm. ### Security Devices located in a datacenter in Europe. Each test runs on a secure and pristine device. ### Real Android and iOS devices Test across real iPhone, iPads, Google, OnePlus, Xiaomi and Samsung phones & tabs. ### Variety of devices Test on any device, flagship or budget, with support for the newest and legacy iOS and Android versions. ### Performance Our infrastructure is built for performance to eliminate test flakiness and latency. The actual fleet ## Every real device you can test on This is the full list, rendered from our live inventory rather than from a marketing number. Every entry is a physical phone or tablet racked in our European datacenter, wiped and re-imaged between sessions. 41 Physical devices 21 iPhone & iPad 20 Android phones & tablets ![Real device testing on Pixel 8](https://testingbot.com/assets/devices/160h/46-d0ca4c1d1985aefc27c52ba2a1c88748875c574d5358e53adcbdeaee4ed078bb.webp) Pixel 8 Android 17.0 ![Real device testing on Pixel 11](https://testingbot.com/assets/devices/160h/45-b90994534fadde53be99a94521c6ea23a39fdf32ad277240be4e2613540b5019.webp) Pixel 11 Android 17.0 ![Real device testing on iPhone 17 Pro](https://testingbot.com/assets/devices/160h/44-6b4402c6996d9b1620879bb1123c40a6b36abb064e65dca0a1b3f05286779e9c.webp) iPhone 17 Pro iOS 26.5 ![Real device testing on iPhone 17 Pro Max](https://testingbot.com/assets/devices/160h/43-59d7e30684332ed4bf25567cbc09ca942eac46d925ccde8352d3e85812480f7d.webp) iPhone 17 Pro Max iOS 26.6 ![Real device testing on iPhone 16 Pro](https://testingbot.com/assets/devices/160h/42-de3704079345026847b739a5b1cf5332595904e54ede66abd988a76fb1e6593c.webp) iPhone 16 Pro iOS 26.4 ![Real device testing on iPad (9th generation)](https://testingbot.com/assets/devices/160h/41-70f7d18afde7620063a49a76cbfd088f8b59c742f3a5f4dd4a513dd6ec5c676c.webp) iPad (9th generation) iOS 26.6 ![Real device testing on Galaxy S26](https://testingbot.com/assets/devices/160h/40-e917ed08e15b5941fed75855ac457dd1df46b471a2020f081c4d70cefb68f4e8.webp) Galaxy S26 Android 16.0 ![Real device testing on Huawei P40](https://testingbot.com/assets/devices/160h/39-32a788c7e66a3da1c88eaaefefd37381f729e510f096b43906968b6907c44c57.webp) Huawei P40 Android 12.0 ![Real device testing on iPhone Air](https://testingbot.com/assets/devices/160h/38-c00a59944eb2e410d375e3f5b39780043f72446bb33b0aea6a26a0c45f309ea1.webp) iPhone Air iOS 26.2 ![Real device testing on Pixel 10](https://testingbot.com/assets/devices/160h/37-738ed465eff0a4bcc33a5294f6f9f4d701fed317ce0e97e202ffc3f9dca7edd4.webp) Pixel 10 Android 16.0 ![Real device testing on iPhone 17](https://testingbot.com/assets/devices/160h/36-06478e9d9935ed794bd3ec02c85ad3a8eafe1de3cc11debd15351c1bdc99005b.webp) iPhone 17 iOS 26.0 ![Real device testing on Pixel 9](https://testingbot.com/assets/devices/160h/35-abb9a661ebbfa84f18bd6985d3e80330ad43fe4ea7d408157c87ae0f560a5ca1.webp) Pixel 9 Android 16.0 ![Real device testing on Galaxy S25](https://testingbot.com/assets/devices/160h/34-ba3caa0fe8fb834ea684e524f6ad690ca4f78402e85a51a48e94ad4dfcb93208.webp) Galaxy S25 Android 15.0 ![Real device testing on iPhone 16](https://testingbot.com/assets/devices/160h/33-c293dfd6742003e86ad82b583f9a5a226773067c2a23d100a4379ae82f6ebba2.webp) iPhone 16 iOS 26.5 ![Real device testing on Redmi Note 13](https://testingbot.com/assets/devices/160h/32-5f416c0f9e99cc3c6782b2b50072f406a8056e009850c16693dbc1fe815f88db.webp) Redmi Note 13 Android 14.0 ![Real device testing on iPhone SE 2022](https://testingbot.com/assets/devices/160h/31-e84c40c6867b3c43060fbaad0e698e01bac7c350cd15c52c23ab159916bbf1e2.webp) iPhone SE 2022 iOS 17.5 ![Real device testing on Galaxy A55](https://testingbot.com/assets/devices/160h/30-ee1932ef83ba5feee5414d6eb777943fb0ba2c6250a83a966761ae70e2bfb4c2.webp) Galaxy A55 Android 14.0 ![Real device testing on Galaxy S24](https://testingbot.com/assets/devices/160h/29-86a13f8a284a09af872058cb0913697ac62b6a1556af0bc9e2bdd796fa4ab0c8.webp) Galaxy S24 Android 14.0 ![Real device testing on Pixel 8](https://testingbot.com/assets/devices/160h/28-d0ca4c1d1985aefc27c52ba2a1c88748875c574d5358e53adcbdeaee4ed078bb.webp) Pixel 8 Android 14.0 ![Real device testing on iPhone 15](https://testingbot.com/assets/devices/160h/27-ac62a400d382791601173f54045c190905de5eb4b927133c0ef5250735228494.webp) iPhone 15 iOS 18.0 ![Real device testing on iPad (8th generation)](https://testingbot.com/assets/devices/160h/26-70f7d18afde7620063a49a76cbfd088f8b59c742f3a5f4dd4a513dd6ec5c676c.webp) iPad (8th generation) iOS 14.7 ![Real device testing on iPhone 11](https://testingbot.com/assets/devices/160h/25-38ef17c9590e56f99d405c33fedc1818a6f6892f4c2e49c5d48a96d87e7aabe7.webp) iPhone 11 iOS 14.8 ![Real device testing on iPhone 14](https://testingbot.com/assets/devices/160h/24-8f5b01be60f4ce141f8f17d7cdd60a54398bcf30a31f6694fe16bfaee5bac195.webp) iPhone 14 iOS 17.6 ![Real device testing on Galaxy S23](https://testingbot.com/assets/devices/160h/23-1a7edd38592e22f8f39e6c9e20ffdb9ae0bfdb809dfe298e4c729b23f51be41c.webp) Galaxy S23 Android 13.0 ![Real device testing on iPhone 13](https://testingbot.com/assets/devices/160h/22-a8bc22314f6a7336f80903f1218ccb695fd45264d67801f00721b28b8acd8c39.webp) iPhone 13 iOS 18.5 ![Real device testing on iPhone XR](https://testingbot.com/assets/devices/160h/19-715ccf8c2a2d82aadc063a2572fc15139e7b36a90d928cc07e1d60f678a85f39.webp) iPhone XR iOS 14.8 ![Real device testing on iPad (8th generation)](https://testingbot.com/assets/devices/160h/18-77056524761232d02b9fb6e93facbf15aac919ab35ebc218db7807c7391690b5.webp) iPad (8th generation) iOS 15.0 ![Real device testing on Galaxy A12](https://testingbot.com/assets/devices/160h/17-f8a23e6f3483a25f35337598005490aa9b206eb372e0f44f9e7ee60d7ae973a4.webp) Galaxy A12 Android 11.0 ![Real device testing on Mi 10](https://testingbot.com/assets/devices/160h/16-d90273d87e3e1eca56fa84606833e77f3d2b1f8cc16c71b1b3cc99dc6a3e0ea1.webp) Mi 10 Android 12.0 ![Real device testing on Galaxy S21](https://testingbot.com/assets/devices/160h/15-751c4b6248cd11c4ebbd49f596ecc2b45522f29405518e024e9b762d36707b9a.webp) Galaxy S21 Android 11.0 ![Real device testing on iPhone 12](https://testingbot.com/assets/devices/160h/14-2ecb8c9b59aed3fcea73e9ec5b95992727fd093ce271e8ddf22492a76f58f330.webp) iPhone 12 iOS 14.5 ![Real device testing on Galaxy S20](https://testingbot.com/assets/devices/160h/12-28c332080f91a69bdef4b18060583dff71ff4d7c0ccaa27fe1ea070527aab737.webp) Galaxy S20 Android 10.0 ![Real device testing on iPhone 11](https://testingbot.com/assets/devices/160h/11-38ef17c9590e56f99d405c33fedc1818a6f6892f4c2e49c5d48a96d87e7aabe7.webp) iPhone 11 iOS 14.0 ![Real device testing on iPhone 8](https://testingbot.com/assets/devices/160h/10-cc4b3c846f9377b2ba147ad2b9c2e1423e31944ce91f55ec4f8792caeb0e1531.webp) iPhone 8 iOS 16.7 ![Real device testing on Galaxy Tab S11](https://testingbot.com/assets/devices/160h/9-909bd0ab1dd2c30af95153fa3cf205ea0ca9275cc472d9e74b40dd4e84471412.webp) Galaxy Tab S11 Android 16.0 ![Real device testing on Galaxy S10](https://testingbot.com/assets/devices/160h/8-9927c1593637288533cf6934d8fa8704bc2c58b941d787f6c22cbb4abbae7ecb.webp) Galaxy S10 Android 10.0 ![Real device testing on Pixel](https://testingbot.com/assets/devices/160h/7-c6df4236e367633a207f65973e04e438e08eff96ce13ea4d403b05998b74f507.webp) Pixel Android 7.1 ![Real device testing on iPhone XR](https://testingbot.com/assets/devices/160h/5-715ccf8c2a2d82aadc063a2572fc15139e7b36a90d928cc07e1d60f678a85f39.webp) iPhone XR iOS 18.7 ![Real device testing on iPad (6th generation)](https://testingbot.com/assets/devices/160h/3-03aa7ada3fcbb792e94cba3190ecc9a0c7635ff03c7298d133f811bf9281a673.webp) iPad (6th generation) iOS 13.5 ![Real device testing on iPhone 6s](https://testingbot.com/assets/devices/160h/2-169433db4e4de6fe1803129e93b1db50498723516d5d3a977a42e19ed6795ff8.webp) iPhone 6s iOS 15.8 ![Real device testing on Galaxy S8](https://testingbot.com/assets/devices/160h/1-121c90cca073f7294901e94938d6169a119328b0ebccc08a701d39603221661d.webp) Galaxy S8 Android 9.0 Need a model that is not listed? A [private device cloud](https://testingbot.com/enterprise/private-device-cloud) lets us buy and rack specific hardware for your account alone. For quick checks on hardware nobody keeps in a rack, we also run hosted [emulators and simulators](https://testingbot.com/features/manual-mobile-testing). Automation frameworks ## Test Automation Frameworks Works instantly with the most popular mobile automation frameworks: - [![Appium Testing](https://testingbot.com/assets/support/appium-4b35a8bb56bae236bd84c539cee9b8bbb489a55ddf48e32d059c86685d230354.svg)](https://testingbot.com/features/automation/appium) ### [Appium](https://testingbot.com/features/automation/appium) Mobile Automation Framework, using WebDriver, runs on physical devices and simulators/emulators. [Try with Appium](https://testingbot.com/features/automation/appium) - [![Espresso Testing](https://testingbot.com/assets/integrations/espresso-a8ff30da0c385f89f4e694bb29fc4779ad132e8e3f4a777a38e608e0492ff610.svg)](https://testingbot.com/features/automation/espresso) ### [Espresso](https://testingbot.com/features/automation/espresso) Android Automation Framework, runs on physical devices and emulators. [Try with Espresso](https://testingbot.com/features/automation/espresso) - [![XCUITest Testing](https://testingbot.com/assets/integrations/xcuitest-4a935f2368661d89d3559b3c5af3377236029478ae76bb43253f3c3bee712096.svg)](https://testingbot.com/features/automation/xcuitest) ### [XCUITest](https://testingbot.com/features/automation/xcuitest) iOS Automation Framework, runs on physical devices and simulators. [Try with XCUITest](https://testingbot.com/features/automation/xcuitest) - [![Maestro Testing](https://testingbot.com/assets/integrations/maestro-9d782f30f13fb4efaf9928b704c7a787eb14cb9e5b20c650137bb65892334b8b.svg)](https://testingbot.com/features/automation/maestro) ### [Maestro](https://testingbot.com/features/automation/maestro) Maestro is a mobile UI testing framework, which supports testing on both iOS and Android. [Try with Maestro](https://testingbot.com/features/automation/maestro) - [![Mobilewright Testing](https://testingbot.com/assets/integrations/mobilewright-fdc12119ed8fb1a08f7d6116eff6fa37f5c4616acccc2b22eca478a707fd3870.png)](https://testingbot.com/support/app-automate/mobilewright) ### [Mobilewright](https://testingbot.com/support/app-automate/mobilewright) Mobilewright runs Playwright-style mobile tests in TypeScript on both iOS and Android. [Try with Mobilewright](https://testingbot.com/support/app-automate/mobilewright) Speed up your suite ## Cut suite time from hours to minutes Drastically shorten your total app test duration by running tests simultaneously. TestingBot provides a variety of devices, ready to run your tests in parallel. [Start free](https://testingbot.com/users/sign_up) ![Cut suite time from hours to minutes](https://testingbot.com/assets/features/selenium/parallel-0229d698e59a8a6b92fe50344a08634624f018331ecf8a849ef83cfb35e635a5.png) ![Video, screenshots & logs](https://testingbot.com/assets/features/app-automated-report-dcef51604982dffa3ee9f02b87a92cef82132f9e21c5b66683738ab69f634ecf.jpg) Debug faster ## Video, screenshots & logs Every mobile test comes with access to generated logs, videos and screenshots. [Start free](https://testingbot.com/users/sign_up) Upload & test in seconds ## TestingBot Storage Upload your native mobile app, as an .apk or .ipa, to [TestingBot Storage](https://testingbot.com/support/api#upload) and start testing immediately. [Start free](https://testingbot.com/users/sign_up) ![TestingBot Storage](https://testingbot.com/assets/features/tb-storage-a1d1845fbed2bac0012e225de7f847431af6ffdb9dd9b6233fd3145ef3681cb9.jpg) An honest comparison ## Real devices vs emulators: when each one is the right call Real device testing is not automatically the correct answer. Emulators and simulators are faster to start, cheaper to run in bulk and perfectly adequate for a large share of everyday checks. We sell both, so here is the split we would actually recommend. ### An emulator or simulator is enough for - 1 Layout, navigation and functional regression on every commit, where you care that the flow works rather than how the hardware behaves. - 2 Broad OS-version sweeps, because spinning up twenty emulator images costs a fraction of occupying twenty physical handsets. - 3 Early development feedback, where a session that starts in seconds matters more than hardware fidelity. - 4 Screen-size and resolution checks, which an emulator reproduces faithfully. ### You need real hardware for - 1 Performance and responsiveness, because an emulator borrows your host machine's CPU and memory and will happily hide a device that is too slow. - 2 Camera, GPS, biometrics, Bluetooth and other sensors, which emulators either stub out or approximate. - 3 Real network behaviour on a real radio, including how the app copes with a genuinely poor mobile connection. - 4 Vendor-specific behaviour, where the manufacturer's own build of Android is what breaks rather than Android itself. Most teams end up running both: [emulators and simulators](https://testingbot.com/features/manual-mobile-testing) on every commit, and real hardware before a release, on the handsets their own analytics say matter most. What you need to buy ## Which plans include real devices Real devices are not a premium tier here. Every TestingBot subscription includes them, and so do the pay-as-you-go minute packs. ### Manual plan Live, hands-on sessions on real iPhones, iPads and Android handsets, driven from your browser. Includes the hosted emulators and simulators too. ### Automated plan Appium, XCUITest, Espresso and Maestro runs on real devices, plus the full desktop browser grid. Codeless AI, visual testing and accessibility scans are included rather than sold separately. ### Pro plan Everything in Manual and Automated on a single subscription, for teams doing both against the same device fleet. ### Pay-as-you-go credits One-time minute packs with no subscription. Minutes cover live manual sessions and automated runs on real devices, desktop browsers, emulators and simulators. Current tiers and parallel-session counts are on the [pricing page](https://testingbot.com/pricing). Signing up is free and needs no card. The 7-day full-access trial, which unlocks every real device, asks for a card and charges nothing during the trial. How we compare ## Real Device Testing: Quick Comparison How TestingBot's real device cloud compares to other major providers. | Capability | TestingBot | BrowserStack | SauceLabs | | --- | --- | --- | --- | | Real iOS & Android devices | | | | | European-hosted devices (data residency) | Yes | US primary | US primary | | GDPR compliant, CSA STAR Level 1 self-assessment | | | | | Appium / XCUITest / Espresso / Maestro | | | No Maestro | | Free for open source | Yes | | | | Dedicated private device cloud | | | | | Secure tunnel for private/staging apps | | | | | Transparent flat-rate pricing | Yes | Custom quote | Custom quote | See more on the [SauceLabs alternative](https://testingbot.com/saucelabs-alternative), [BrowserStack alternative](https://testingbot.com/browserstack-alternative) and [LambdaTest alternative](https://testingbot.com/lambdatest-alternative) pages. Real-world conditions ## Test native device features Access and modify device features to test your app in various real-world conditions. ### Network Simulation Test for scenarios like connectivity, fluctuating network & many more. ### Localization testing Test your apps across different timezones, languages and geolocations. ### Dark Mode Test your app's UI with a dark background to ensure a smooth user experience. ### App Orientation Test your app in the landscape or portrait mode. Switch between orientations to test your app's response. ### Audio Record and hear in realtime the audio generated on the device. ### Settings Control various settings on the device. Private device cloud ## Your very own Mobile Devices Choose your own iOS and Android devices. We will purchase these devices, install and configure them in our secure datacenter. These devices will be only available to you, 24x7. Private devices can be customised to your needs. Have us install specific profiles and apps, or change specific settings of the device. You decide. [Learn more](https://testingbot.com/enterprise/private-device-cloud) ![Your very own Mobile Devices](https://testingbot.com/assets/enterprise/datacenter-d4fc366e9a4e8a106ec41e6817e21adb2aeb366a397ac258fd29ba9cd4de72c4.jpg) Secure Tunnel ## Test Staged Websites Use the secure TestingBot Tunnel to easily test your staging and development environments across all browsers, ensuring seamless access behind firewalls or private networks. Test data and websites from behind your firewall on the TestingBot cloud. ![TestingBot Tunnel for real devices](https://testingbot.com/assets/features/tunnel-diagram-poster-8c25e19e6fe4f7276a46a96b31a0ad0e07bb4f5bf98dc192d2b70897cfbb1a26.png) [Read more](https://testingbot.com/support/tunnel) FAQ ## Frequently Asked Questions Common questions about real device testing on TestingBot. What is real device testing? Real device testing is the practice of running automated or manual tests on physical smartphones and tablets, as opposed to emulators or simulators. It surfaces hardware-specific issues like CPU/RAM constraints, battery behaviour, real network conditions and device-specific rendering quirks that emulators or simulators cannot reproduce. Should I use a real device or an emulator? Use emulators for fast feedback during early development and unit-style flows. Use real devices for release validation, performance testing, biometric flows, camera/sensor logic, push notifications and final acceptance testing. Most teams run both: emulators in CI on every commit, real devices on the release branch. Where are TestingBot's real devices located? All TestingBot devices are hosted in a secure datacenter in Belgium, Europe. Every device is wiped and re-imaged after each test session. None are jailbroken or rooted. Which test frameworks does the real device cloud support? Appium (iOS and Android), XCUITest (iOS), Espresso (Android), Maestro (iOS and Android), Mobilewright (iOS and Android) and any WebDriver-compatible framework. You can also drive devices manually from your browser, or use TestingBot's codeless AI test creator to generate tests from plain English. Can I test apps that are not yet on the App Store or Google Play? Yes. Upload an .ipa or .apk build to TestingBot Storage and install it on any real device. You can also test apps already published to the App Store or Google Play, or connect to staging environments behind a corporate firewall using TestingBot Tunnel. What's the difference between the public and private device cloud? Public-cloud devices are shared across customers and are wiped between sessions. Private-cloud devices are dedicated phones we purchase, configure and host exclusively for your account, with optional persistence between tests, custom profiles and pre-installed apps. How does TestingBot compare to BrowserStack and SauceLabs for real device testing? TestingBot offers European-hosted real devices (BrowserStack and SauceLabs are primarily US-hosted), GDPR and CSA STAR compliance, transparent flat-rate pricing, free for open source, and identical Appium/XCUITest/Espresso APIs. Customers usually migrate for the EU data residency, faster device allocation and direct support from the engineering team. Looking for hands-on details? See the [live mobile app testing](https://testingbot.com/features/manual-mobile-testing) page or browse our [supported devices](https://testingbot.com/support/app-automate/devices). Start in seconds ### Sign up for a Free Trial Start testing your apps with TestingBot. [Start a free trial](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) Resources you may like [![Appium Flutter Driver: Testing Flutter Apps](https://testingbot.com/assets/resources/articles/13-77f190ec948df41b31594f8fc224348844ebc412f374eb6f18c9d4215b4fa49b.webp)](https://testingbot.com/resources/articles/appium-flutter-app-testing) ### [Appium Flutter Driver: Testing Flutter Apps](https://testingbot.com/resources/articles/appium-flutter-app-testing) How to test a Flutter app with Appium: what the framework compiles to, how the Appium Flutter Driver works, and how to build an app for testing. [Read more: Appium Flutter Driver: Testing Flutter Apps](https://testingbot.com/resources/articles/appium-flutter-app-testing) [![How to do localisation testing for mobile apps](https://testingbot.com/assets/resources/articles/12-9db6c7be671827ade1212fa6e476ee36e3365f01569100ee46fe22405c035760.webp)](https://testingbot.com/resources/articles/localized-testing-mobile-app) ### [How to do localisation testing for mobile apps](https://testingbot.com/resources/articles/localized-testing-mobile-app) Find out how to perform localisation testing with mobile apps. Change your language or locale with Appium, XCUITest and Espresso. [Read more: How to do localisation testing for mobile apps](https://testingbot.com/resources/articles/localized-testing-mobile-app) [![Dark Mode Testing with Appium](https://testingbot.com/assets/resources/articles/22-ed15fa0f3fde7566416fff5078b44689dc274439b2fb829f8c68603e909a65d6.webp)](https://testingbot.com/resources/articles/dark-mode-testing) ### [Dark Mode Testing with Appium](https://testingbot.com/resources/articles/dark-mode-testing) How to switch a native app between light and dark mode from an Appium test on iOS and Android, and the layout bugs dark mode tends to expose. [Read more: Dark Mode Testing with Appium](https://testingbot.com/resources/articles/dark-mode-testing) --- URL: https://testingbot.com/playwright-vs-selenium Framework comparison · 2026 Last updated 2026-08-21 # Playwright vs Selenium Playwright and Selenium are the two dominant browser automation frameworks. This guide compares them head-to-head on speed, language support, browser coverage, debugging, and which to pick, with code samples and a verdict you can act on in five minutes. [Run both on TestingBot](https://testingbot.com/users/sign_up) [Jump to comparison](https://testingbot.com#comparison) - **Browsers & devices:** 6100+ - **Cloud parallelism:** 100× - **Uptime SLA:** 99.9% ## Trusted by some of the world's most innovative companies ## On this page 1. [Which one should you pick?](https://testingbot.com#which-should-you-pick) 2. [What are Playwright and Selenium?](https://testingbot.com#what-are-playwright-and-selenium) 3. [Playwright vs Selenium: side-by-side comparison](https://testingbot.com#comparison) 4. [Filling a form, asserting the result](https://testingbot.com#code-comparison) 5. [Where Cypress fits](https://testingbot.com#where-cypress-fits) 6. [When to choose which](https://testingbot.com#when-to-choose-which) 7. [Stop choosing: run both on the same grid](https://testingbot.com#run-both) 8. [Frequently Asked Questions](https://testingbot.com#faq) 30-second answer ## Which one should you pick? ![Playwright](https://testingbot.com/assets/support/playwright-2dcb4e75b55ac0d1135cfa47be6137b13f46a2047ce78b96a34f51b0e2cc0d96.svg) Pick Playwright if… You're starting a new web project, your team writes JavaScript or TypeScript, and you want fast tests, auto-waiting and built-in tracing out of the box. ![Selenium](https://testingbot.com/assets/support/selenium-266ecc0599068fc6eca0bcd9174fb46053c95cebe1c66f7a50c480f3f1a52929.svg) Pick Selenium if… Your team writes Java, C# or Ruby, you need legacy browser support (Internet Explorer, older Safari), or you have an existing Selenium suite and a strong Page Object library. Pick both if… You have separate frontend and backend test suites, you're migrating gradually, or you want Playwright for new code and Selenium for the long-tail of legacy browsers. TestingBot runs both on the same grid. Background ## What are Playwright and Selenium? Two frameworks, two philosophies. The same goal: drive a real browser to automate end-to-end testing. ![Playwright](https://testingbot.com/assets/support/playwright-2dcb4e75b55ac0d1135cfa47be6137b13f46a2047ce78b96a34f51b0e2cc0d96.svg) ### Playwright Released 2020 · Microsoft · Apache 2.0 Playwright is an end-to-end testing framework built by Microsoft that controls Chromium, Firefox and WebKit through a single API. It was designed by the team that originally built Puppeteer at Google, with cross-browser support and modern web app needs in mind. It uses the browser's DevTools protocol (or its own bridge for Firefox/WebKit) instead of WebDriver, which lets it auto-wait for elements, intercept network traffic and capture rich traces by default. - TypeScript / JavaScript / Python / Java / C# bindings - Auto-wait, network interception, fixtures, traces - Chromium, Firefox, WebKit · no IE / legacy Safari ![Selenium](https://testingbot.com/assets/support/selenium-266ecc0599068fc6eca0bcd9174fb46053c95cebe1c66f7a50c480f3f1a52929.svg) ### Selenium Released 2004 · Open Source · Apache 2.0 Selenium is the original browser automation framework and the foundation of the W3C WebDriver standard. Selenium WebDriver drives any browser through a vendor-supplied driver (chromedriver, geckodriver, safaridriver, edgedriver, IEDriverServer) using the same wire protocol. Twenty years of ecosystem maturity means deep IDE integration, mature Page Object patterns, and bindings for every major language. Selenium 4 added Chrome DevTools support, BiDi events and bidirectional protocols. - Java / Python / C# / Ruby / JavaScript / Kotlin - W3C WebDriver standard, works with every browser - Chrome, Firefox, Safari, Edge, IE 11: full coverage Head to head ## Playwright vs Selenium: side-by-side comparison Across the dimensions that matter for picking, migrating or running both in CI. | Dimension | Playwright | Selenium | Cypress | | --- | --- | --- | --- | | First release | 2020 | 2004 | 2017 | | Maintained by | Microsoft | Open-source community + W3C | Cypress.io | | Languages | TS, JS, Python, Java, C# | Java, Python, C#, Ruby, JS, Kotlin | JavaScript / TypeScript only | | Browsers | Chromium, Firefox, WebKit | Chrome, Firefox, Safari, Edge, IE 11 | Chrome, Edge, Firefox, WebKit (experimental) | | Mobile testing | Mobile emulation only | Real iOS + Android via Appium | Viewport emulation only | | Wire protocol | CDP + custom (Firefox/WebKit) | W3C WebDriver standard | Executes inside the browser | | Auto-waiting | Built in everywhere | Manual waits or WebDriverWait | Built in everywhere | | Network interception | First-class API | BiDi (Selenium 4) or proxy | First-class (cy.intercept) | | Multiple tabs / origins | Native, same context | Window switching API | No multi-tab; cy.origin for cross-origin | | Test runner | Built-in (@playwright/test) | Bring your own (JUnit, pytest, Mocha) | Built-in | | Parallel execution | Built-in worker pool | Selenium Grid or test-runner level | Cypress Cloud or CI-level sharding | | Trace / replay | Built-in Trace Viewer | Screenshots + manual logs | Time-travel debugger + video | | Debugging | Inspector + UI mode | IDE + driver logs | Interactive test runner | | Speed (typical suite) | About 30-40% faster on Chromium | Mature, predictable | Fast, but one browser per run | | Ecosystem maturity | Growing fast since 2020 | 20 years of integrations | Large plugin ecosystem | | Page Object pattern | Supported, less common | Industry standard | Supported, discouraged by the docs | | Runs on TestingBot | ✓ | ✓ | ✓ | Speed numbers from public benchmarks; vary by application. Both frameworks run on TestingBot's cloud with identical APIs to local execution. The same test in both ## Filling a form, asserting the result A login flow with auto-wait (Playwright) versus an explicit wait (Selenium). Both pass on the same TestingBot grid. Playwright + TypeScript login.spec.ts ```typescript // playwright.config.ts → set TestingBot endpoint import { test, expect } from '@playwright/test'; test('login redirects to dashboard', async ({ page }) => { await page.goto('https://app.example.com/login'); // auto-wait, no explicit wait needed await page.getByLabel('Username').fill('jane@example.com'); await page.getByLabel('Password').fill('••••••••'); await page.getByRole('button', { name: 'Sign in' }).click(); await expect(page).toHaveURL(/\/dashboard/); await expect(page.getByRole('heading', { name: /welcome, jane/i })).toBeVisible(); }); ``` Selenium + Python test\_login.py ```python # Driver points at TestingBot remote URL from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC def test_login_redirects_to_dashboard(driver): driver.get('https://app.example.com/login') driver.find_element(By.NAME, 'username').send_keys('jane@example.com') driver.find_element(By.NAME, 'password').send_keys('••••••••') driver.find_element(By.CSS_SELECTOR, 'button[type=submit]').click() # explicit wait WebDriverWait(driver, 10).until(EC.url_contains('/dashboard')) welcome = driver.find_element(By.TAG_NAME, 'h1') assert 'Welcome, Jane' in welcome.text ``` Both tests are pointed at the same [Playwright](https://testingbot.com/features/automation/playwright) / [Selenium](https://testingbot.com/features/automation/selenium) endpoint on TestingBot's cloud. Same browsers, same parallelism, same dashboard. The third option ## Where Cypress fits Most teams weighing Playwright against Selenium look at Cypress too. Here is the short version. Cypress runs your test code inside the browser alongside the application, rather than driving it from the outside. That is what gives it the interactive test runner and time-travel debugging people like, and it is also the source of its limits: JavaScript and TypeScript only, no native mobile app testing, no multiple browser tabs, and cross-origin work needs cy.origin. Against Playwright, the trade is developer experience versus reach. Playwright covers more languages and more browser engines and has no trouble with tabs or origins. Against Selenium, the trade is ergonomics versus coverage: Selenium remains the only one of the three that drives real iOS and Android devices, through Appium. If Cypress is genuinely on your shortlist, the dedicated [Cypress vs Playwright](https://testingbot.com/cypress-vs-playwright) and [Cypress vs Selenium](https://testingbot.com/cypress-vs-selenium) comparisons go through it properly. All three run on TestingBot. Decision matrix ## When to choose which ### Choose Playwright when - You're starting a green-field test suite, no migration cost to weigh. - Your stack is JavaScript / TypeScript-first and you want the test runner that ships with the framework. - You need network mocking, multi-tab flows, file uploads or geolocation overrides without bolt-on tools. - You want trace-based debugging; record, save, scrub through a failed run frame-by-frame. - Your target browsers are Chromium, Firefox and WebKit. No Internet Explorer 11 or older Safari requirement. [Run Playwright on TestingBot](https://testingbot.com/features/automation/playwright) ### Choose Selenium when - Your team writes Java, C# or Ruby. Playwright's bindings exist but the JS-first ergonomics get diluted. - You need to test on Internet Explorer 11, older Safari, or unusual browser builds where only WebDriver works. - You have a mature Page Object library, custom locators, or BDD framework (Cucumber, SpecFlow) you don't want to throw away. - You're standardising across web AND real mobile devices. Selenium's API is the closest thing to a unified WebDriver story (with Appium for mobile). - Stability and ecosystem maturity outweigh fewer features. Selenium has been battle-tested for 20 years. [Run Selenium on TestingBot](https://testingbot.com/features/automation/selenium) TestingBot supports both ## Stop choosing: run both on the same grid Point Playwright at the TestingBot endpoint, point Selenium at the same one, and your tests share the same 6100+ browsers and devices, the same dashboard, the same parallel slots and the same EU data residency. - Same auth, same project, same billing - Side-by-side test history for both frameworks - Free for open source, both frameworks [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) testingbot.com / endpoint // Playwright use: { wsEndpoint: 'wss://cloud.testingbot.com/playwright' } # Selenium webdriver.Remote( command\_executor='https://hub.testingbot.com/wd/hub' ) FAQ ## Frequently Asked Questions The questions teams ask before picking these frameworks, or migrating between them. Is Playwright faster than Selenium? Generally yes, on Chromium-based browsers. Typical suites run 30–40% faster on Playwright thanks to the DevTools protocol bypassing WebDriver's HTTP round-trips. The gap closes for Firefox and Safari (where Playwright uses its own bridges) and disappears once your tests are network-bound. Selenium 4's BiDi support narrows the gap further. For most teams the speed difference is real but not the deciding factor. Should I migrate from Selenium to Playwright? Only if the cost is justified. If your existing Selenium suite is stable, in a non-JavaScript language, and uses Page Objects extensively, migrating is rarely worth the effort. If your suite is flaky, JS-based and lacks tracing, Playwright's auto-wait and Trace Viewer alone usually pay for the migration. Pragmatic teams keep both: Selenium for what's already working, Playwright for new test suites. Does Playwright support all browsers Selenium does? No. Playwright supports Chromium (and Chrome/Edge by extension), Firefox and WebKit (which is close to Safari but not identical). Selenium drives the actual browser builds, including Internet Explorer 11, older Safari versions and rare browsers, using vendor-shipped drivers. If you need to test on IE 11 or a specific legacy Safari, Selenium is your only option. Can Playwright test mobile apps? Not natively. Playwright offers mobile browser emulation (you can simulate a mobile viewport and user agent in Chromium), but it cannot drive native iOS or Android apps. For real mobile app testing you need [Appium](https://testingbot.com/features/automation/appium), [XCUITest](https://testingbot.com/features/automation/xcuitest), [Espresso](https://testingbot.com/features/automation/espresso) or [Maestro](https://testingbot.com/features/automation/maestro). Selenium can drive mobile via Appium, which is built on the WebDriver protocol. TestingBot supports all of these on real iOS and Android devices. Which is better for cross-browser testing? For modern Chromium / Firefox / WebKit, Playwright's single API gives you faster, more reliable cross-browser runs out of the box. For comprehensive coverage including legacy browsers (IE 11, older Safari, Edge Legacy), Selenium is still the only option that drives the real browser binaries. Either way, running across many real browsers in parallel benefits hugely from using TestingBot's cloud. Both frameworks integrate with TestingBot's cloud with a one-line config change. Do I need to learn JavaScript to use Playwright? No, but Playwright's ergonomics are JS/TS-first. The Python, Java and C# bindings are first-class but lag the JS API on new features by weeks. If your team already writes Python or Java, Playwright is still a strong choice, but Selenium's binding parity makes it the more natural fit for non-JS stacks. Can I run Playwright and Selenium on TestingBot? Yes, both run on the same TestingBot cloud. Playwright connects via the wsEndpoint config to `wss://cloud.testingbot.com/playwright`, Selenium connects to `https://hub.testingbot.com/wd/hub` via `webdriver.Remote`. Tests show up in the same dashboard, share parallel slots, and benefit from the same EU data residency, video recording, live debugging and CI/CD integrations. Both are free for open source projects. Is Playwright going to replace Selenium? Not wholesale. Playwright is taking a large share of new JavaScript and TypeScript projects, but Selenium is a W3C standard with 20 years of installed base, bindings for six languages, and the only route to real iOS and Android devices via Appium. The realistic outcome is coexistence: new suites lean Playwright, large existing suites and non-JS stacks stay on Selenium. Plenty of teams run both. Is Selenium outdated now? No. Selenium 4 moved to the W3C WebDriver standard and added relative locators, a Chrome DevTools Protocol bridge and BiDi support. It is slower to write and needs explicit waits more often than Playwright, but it is actively maintained and remains the broadest option for language and browser coverage, including Safari, Internet Explorer 11 and real mobile devices. Can I learn Playwright without Selenium? Yes. Playwright does not build on Selenium and shares none of its API, so there is nothing you need to unlearn or learn first. Prior Selenium experience helps mainly with general concepts such as locators, waits and the Page Object pattern, not with Playwright syntax. If you are starting fresh, learning Playwright directly is the shorter path. Does Playwright require coding? Yes. Playwright is a code-first framework: tests are written in TypeScript, JavaScript, Python, Java or C#. The Codegen tool records your browser actions and generates a working test script, which lowers the barrier considerably, but the output is still code you edit and maintain. There is no no-code mode. Going deeper? See the dedicated [Playwright](https://testingbot.com/features/automation/playwright) and [Selenium](https://testingbot.com/features/automation/selenium) pages. ## Related comparisons [Cypress vs Selenium](https://testingbot.com/cypress-vs-selenium) [Cypress vs Playwright](https://testingbot.com/cypress-vs-playwright) ### Sign up for a Free Trial Run Playwright, Selenium, or both, on TestingBot's cloud. [Start a free trial](https://testingbot.com/users/sign_up) Resources you may like [![Visual Testing with Playwright](https://testingbot.com/assets/resources/articles/14-f3f2e8954c8a47cf0bb29059b44f4d2f40998442e22db6f8acdccb8e5b34d8c1.webp)](https://testingbot.com/resources/articles/playwright-visual-regression-testing) ### [Visual Testing with Playwright](https://testingbot.com/resources/articles/playwright-visual-regression-testing) Playwright provides automated browser testing. It offers a built-in feature to perform visual regression testing for your website. [Read more: Visual Testing with Playwright](https://testingbot.com/resources/articles/playwright-visual-regression-testing) [![Selenium and Generative AI](https://testingbot.com/assets/resources/articles/28-af707bf052844eb00f861bb6ebba57e6be4479dde1c78a5df4e70d2f597a8b18.webp)](https://testingbot.com/resources/articles/generative-ai-selenium) ### [Selenium and Generative AI](https://testingbot.com/resources/articles/generative-ai-selenium) How to use generative AI to produce realistic test data for Selenium runs, and why varied data uncovers bugs that fixed fixtures never will. [Read more: Selenium and Generative AI](https://testingbot.com/resources/articles/generative-ai-selenium) [![Playwright Codegen: Record Tests in VS Code](https://testingbot.com/assets/resources/articles/32-277318b88c5e35eb2c96f03d9b53a1ef8eb2d96db184679892b5008b0bad9d0f.webp)](https://testingbot.com/resources/articles/playwright-recorder) ### [Playwright Codegen: Record Tests in VS Code](https://testingbot.com/resources/articles/playwright-recorder) How to record Playwright scripts from Visual Studio Code rather than writing them by hand, then run the recorded tests on a browser grid. [Read more: Playwright Codegen: Record Tests in VS Code](https://testingbot.com/resources/articles/playwright-recorder) --- URL: https://testingbot.com/cypress-vs-selenium Framework comparison · 2026 # Cypress vs Selenium Cypress and Selenium take opposite approaches to browser testing: one runs inside the browser, the other drives it from outside. This guide compares them head-to-head on architecture, languages, browser coverage, parallelisation and debugging, with code samples and a verdict you can act on in five minutes. [Run both on TestingBot](https://testingbot.com/users/sign_up) [Jump to comparison](https://testingbot.com#comparison) - **Browsers & devices:** 6100+ - **Cloud parallelism:** 100× - **Uptime SLA:** 99.9% ## Trusted by some of the world's most innovative companies 30-second answer ## Which one should you pick? ![Cypress](https://testingbot.com/assets/support/cypress-409c9fd3f79115ed7676aa1df4f6a217dc4402b3366a61984924aec7b4fefa97.svg) Pick Cypress if… Your team writes JavaScript or TypeScript, you test a modern web app on Chrome, Edge or Firefox, and you want the best developer experience with built-in retries, time-travel debugging and a test runner included. ![Selenium](https://testingbot.com/assets/support/selenium-266ecc0599068fc6eca0bcd9174fb46053c95cebe1c66f7a50c480f3f1a52929.svg) Pick Selenium if… Your team writes Java, C# or Ruby, you need legacy browser support (Internet Explorer, Safari), real mobile devices via Appium, or multi-tab and cross-origin flows that Cypress cannot handle. Pick both if… You want Cypress for fast component and frontend tests and Selenium for broad cross-browser and mobile coverage. TestingBot runs both on the same grid, in parallel, in one dashboard. Background ## What are Cypress and Selenium? Two frameworks, two architectures. The same goal: drive a browser to automate end-to-end testing. ![Cypress](https://testingbot.com/assets/support/cypress-409c9fd3f79115ed7676aa1df4f6a217dc4402b3366a61984924aec7b4fefa97.svg) ### Cypress Released 2017 · Cypress.io · MIT Cypress is a JavaScript end-to-end testing framework that runs your tests inside the browser, in the same run loop as your application. That architecture gives it direct access to the DOM, automatic waiting and time-travel snapshots out of the box. Because it lives in the browser, Cypress has an excellent developer experience: a built-in test runner, real-time reloads and a visual debugger. The same architecture also limits it to JavaScript and TypeScript, a fixed set of browsers, and single-tab flows. - JavaScript and TypeScript only - Auto-retries, time-travel debugging, cy.intercept - Chrome, Edge, Firefox, Electron · no IE / Safari ![Selenium](https://testingbot.com/assets/support/selenium-266ecc0599068fc6eca0bcd9174fb46053c95cebe1c66f7a50c480f3f1a52929.svg) ### Selenium Released 2004 · Open Source · Apache 2.0 Selenium is the original browser automation framework and the foundation of the W3C WebDriver standard. Selenium WebDriver drives any browser from the outside through a vendor-supplied driver (chromedriver, geckodriver, safaridriver, edgedriver) using the same wire protocol. Twenty years of ecosystem maturity means bindings for every major language, deep IDE integration, mature Page Object patterns, and real mobile device testing through Appium, which is built on the same protocol. - Java / Python / C# / Ruby / JavaScript / Kotlin - W3C WebDriver standard, works with every browser - Chrome, Firefox, Safari, Edge, IE 11 · real mobile via Appium Head to head ## Cypress vs Selenium: side-by-side comparison Across the dimensions that matter for picking, migrating or running both in CI. Cypress versus Selenium feature comparison| Dimension | Cypress | Selenium | | --- | --- | --- | | First release | 2017 | 2004 | | Maintained by | Cypress.io | Open-source community + W3C | | Languages | JavaScript / TypeScript only | Java, Python, C#, Ruby, JS, Kotlin | | Architecture | Runs inside the browser | Drives the browser via WebDriver | | Browsers | Chrome, Edge, Firefox, Electron | Chrome, Firefox, Safari, Edge, IE 11 | | Mobile testing | Viewport emulation only | Real iOS + Android via Appium | | Auto-waiting | Built-in retries | Manual waits or WebDriverWait | | Network stubbing | First-class (cy.intercept) | BiDi (Selenium 4) or proxy | | Multiple tabs / windows | Not supported | Window switching API | | Cross-origin | cy.origin() (constrained) | Native | | Test runner | Built in | Bring your own (JUnit, pytest, Mocha…) | | Debugging | Time-travel snapshots + runner | IDE + driver logs | | Parallel execution | Cypress Cloud (paid) or a grid | Selenium Grid | | Developer experience | Real-time reload, in-browser | Functional, IDE-based | | Ecosystem maturity | Growing fast since 2017 | 20 years of integrations | | Free for open source on TestingBot | ✓ | ✓ | Feature notes reflect Cypress 13.x and Selenium 4.x as of 2026. Both frameworks run on TestingBot's cloud, Cypress through the testingbot-cypress-cli and Selenium through the WebDriver hub. The same test in both ## Logging in, asserting the result A login flow with built-in retries (Cypress) versus an explicit wait (Selenium). Both run on the same TestingBot grid. Cypress + TypeScript login.cy.ts ```typescript // run via: testingbot-cypress run describe('login', () => { it('redirects to the dashboard', () => { cy.visit('https://app.example.com/login'); // auto-retries until actionable cy.get('#username').type('jane@example.com'); cy.get('#password').type('••••••••'); cy.contains('button', 'Sign in').click(); cy.url().should('include', '/dashboard'); cy.contains('h1', /welcome, jane/i).should('be.visible'); }); }); ``` Selenium + Python test\_login.py ```python # Driver points at TestingBot remote URL from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC def test_login_redirects_to_dashboard(driver): driver.get('https://app.example.com/login') driver.find_element(By.NAME, 'username').send_keys('jane@example.com') driver.find_element(By.NAME, 'password').send_keys('••••••••') driver.find_element(By.CSS_SELECTOR, 'button[type=submit]').click() # explicit wait WebDriverWait(driver, 10).until(EC.url_contains('/dashboard')) welcome = driver.find_element(By.TAG_NAME, 'h1') assert 'Welcome, Jane' in welcome.text ``` The Cypress spec runs through the [testingbot-cypress-cli](https://testingbot.com/support/web-automate/cypress); the Selenium test connects to the [WebDriver hub](https://testingbot.com/support/web-automate/selenium). Both appear in the same TestingBot dashboard. Decision matrix ## When to choose which ### Choose Cypress when - Your stack is JavaScript or TypeScript-first and you want the test runner, assertions and retries to ship with the framework. - You are testing a modern web app on Chrome, Edge or Firefox, with no Internet Explorer or Safari requirement. - You value developer experience: time-travel debugging, real-time reloads and readable failure snapshots. - You want first-class network stubbing with cy.intercept, without bolting on a proxy. - Your flows stay within a single tab and mostly a single origin (cy.origin helps, within limits). [Run Cypress on TestingBot](https://testingbot.com/features/automation/cypress) ### Choose Selenium when - Your team writes Java, C#, Ruby or Python and you want full binding parity, not a JavaScript-first API. - You need to test on Internet Explorer 11, Safari, or browser builds Cypress does not support. - You test real iOS and Android devices, where Appium reuses the WebDriver protocol Selenium is built on. - Your tests open multiple tabs or windows, or cross several origins, which Cypress handles only partially. - You have a mature Page Object library or BDD framework (Cucumber, SpecFlow) you do not want to rewrite. [Run Selenium on TestingBot](https://testingbot.com/features/automation/selenium) TestingBot supports both ## Stop choosing, run both on the same grid Run Cypress through the testingbot-cypress-cli and point Selenium at the WebDriver hub. Your tests share the same 6100+ browsers and devices, the same dashboard, the same parallel slots and the same EU data residency, with no Cypress Cloud subscription required. - Cypress parallelisation without a Cypress Cloud plan - Side-by-side test history for both frameworks - Free for open source, both frameworks [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) testingbot.com / run both // Cypress $ npm i -D testingbot-cypress-cli $ testingbot-cypress run # reads testingbot.json # Selenium webdriver.Remote( command\_executor='https://hub.testingbot.com/wd/hub' ) FAQ ## Frequently Asked Questions The questions teams ask before picking, or migrating between, these frameworks. Is Cypress better than Selenium? Neither is universally better; they suit different teams. Cypress wins on developer experience for JavaScript and TypeScript teams testing modern web apps: built-in retries, time-travel debugging and a test runner included. Selenium wins on breadth: more languages, every browser including Internet Explorer and Safari, real mobile via Appium, and multi-tab and cross-origin flows. Pick Cypress for ergonomics within its constraints; pick Selenium for coverage and flexibility. Should I migrate from Selenium to Cypress? Only if your needs fit within Cypress's constraints. If your suite is JavaScript-based, targets Chrome, Edge or Firefox, and stays within a single tab, Cypress's retries and debugging often justify the move. If you rely on Java, C# or Ruby, test Internet Explorer or Safari, drive real mobile devices, or need multi-tab and cross-origin flows, migrating will cost you capabilities. Many teams keep both: Cypress for frontend tests, Selenium for broad cross-browser and mobile coverage. Does Cypress support all the browsers Selenium does? No. Cypress runs on Chromium-based browsers (Chrome, Edge, Electron) and Firefox, with experimental WebKit support. It cannot drive Internet Explorer or real Safari. Selenium drives the actual browser binaries through vendor drivers, including Internet Explorer 11 and Safari. If you need IE or Safari coverage, Selenium is the only option of the two. Can Cypress test mobile apps? Not natively. Cypress offers viewport resizing to emulate a mobile screen, but it cannot drive native iOS or Android apps. For real mobile testing you need [Appium](https://testingbot.com/features/automation/appium), [XCUITest](https://testingbot.com/features/automation/xcuitest), [Espresso](https://testingbot.com/features/automation/espresso) or [Maestro](https://testingbot.com/features/automation/maestro). Appium reuses the WebDriver protocol Selenium is built on. TestingBot runs all of these on real iOS and Android devices. Do I need Cypress Cloud to run Cypress in parallel? No. Cypress's own parallelisation and recording run through Cypress Cloud, a paid subscription. TestingBot is an alternative: the testingbot-cypress-cli zips your specs, uploads them and runs them in parallel across Chrome, Edge and Firefox on the cloud grid, with live logs and video, and no Cypress Cloud plan. You set the parallel count in testingbot.json. Can Cypress and Selenium share the same CI pipeline? Yes. Both run in CI/CD and report results back to the same place on TestingBot. A common setup runs Cypress for fast frontend specs and Selenium for broad cross-browser and mobile coverage in the same pipeline, both pointed at the TestingBot grid with the same credentials and build name. Can I run Cypress and Selenium on TestingBot? Yes, both run on the same TestingBot cloud. Cypress runs through the `testingbot-cypress-cli` npm package, configured with a `testingbot.json` file and started with `testingbot-cypress run`. Selenium connects to `https://hub.testingbot.com/wd/hub` via `webdriver.Remote`. Tests from both show up in the same dashboard, share parallel slots, and benefit from the same EU data residency, video recording and CI/CD integrations. Both are free for open source projects. Going deeper? See the dedicated [Cypress](https://testingbot.com/features/automation/cypress) and [Selenium](https://testingbot.com/features/automation/selenium) pages. ## Related comparisons [Cypress vs Playwright](https://testingbot.com/cypress-vs-playwright) [Playwright vs Selenium](https://testingbot.com/playwright-vs-selenium) ### Sign up for a Free Trial Run Cypress, Selenium, or both, on TestingBot's cloud. No Cypress Cloud subscription required. [Start a free trial](https://testingbot.com/users/sign_up) Resources you may like [![Tutorial on debugging Cypress tests](https://testingbot.com/assets/resources/articles/10-7ae9f6d9435bea1c54ec815ad535dc4a590086e6165263a9eba3441635d55726.webp)](https://testingbot.com/resources/articles/how-to-debug-cypress-tests) ### [Tutorial on debugging Cypress tests](https://testingbot.com/resources/articles/how-to-debug-cypress-tests) How to debug a failing Cypress test using the debugger, console logs, the stack trace, and the debug and pause commands. [Read more: Tutorial on debugging Cypress tests](https://testingbot.com/resources/articles/how-to-debug-cypress-tests) [![Selenium and Generative AI](https://testingbot.com/assets/resources/articles/28-af707bf052844eb00f861bb6ebba57e6be4479dde1c78a5df4e70d2f597a8b18.webp)](https://testingbot.com/resources/articles/generative-ai-selenium) ### [Selenium and Generative AI](https://testingbot.com/resources/articles/generative-ai-selenium) How to use generative AI to produce realistic test data for Selenium runs, and why varied data uncovers bugs that fixed fixtures never will. [Read more: Selenium and Generative AI](https://testingbot.com/resources/articles/generative-ai-selenium) [![Cypress and Cucumber Browser Testing](https://testingbot.com/assets/resources/articles/6-495c5e14cb4bf1e282858fc7e2cfcbe89775e3790b88d34c15bf9a6f36d387d4.webp)](https://testingbot.com/resources/articles/cypress-cucumber-testing) ### [Cypress and Cucumber Browser Testing](https://testingbot.com/resources/articles/cypress-cucumber-testing) How to combine Cypress with Cucumber so end-to-end tests read as plain language, including a worked example and the TypeScript setup. [Read more: Cypress and Cucumber Browser Testing](https://testingbot.com/resources/articles/cypress-cucumber-testing) --- URL: https://testingbot.com/cypress-vs-playwright Framework comparison · 2026 # Cypress vs Playwright Cypress and Playwright are the two leading modern JavaScript end-to-end testing frameworks. This guide compares them head-to-head on architecture, browser coverage, languages, parallel execution and debugging, with code samples and a verdict you can act on in five minutes. [Run both on TestingBot](https://testingbot.com/users/sign_up) [Jump to comparison](https://testingbot.com#comparison) - **Browsers & devices:** 6100+ - **Cloud parallelism:** 100× - **Uptime SLA:** 99.9% ## Trusted by some of the world's most innovative companies 30-second answer ## Which one should you pick? ![Cypress](https://testingbot.com/assets/support/cypress-409c9fd3f79115ed7676aa1df4f6a217dc4402b3366a61984924aec7b4fefa97.svg) Pick Cypress if… You want the most polished developer experience: an in-browser test runner, time-travel debugging, automatic retries and first-class component testing, and your targets are Chrome, Edge and Firefox. ![Playwright](https://testingbot.com/assets/support/playwright-2dcb4e75b55ac0d1135cfa47be6137b13f46a2047ce78b96a34f51b0e2cc0d96.svg) Pick Playwright if… You need WebKit (Safari engine) coverage, more than one language (TypeScript, Python, Java, C#), multi-tab or multi-origin flows, or free built-in parallel execution without a paid dashboard. Pick both if… You have an existing Cypress suite you like and want Playwright for new cross-browser or multi-tab scenarios. TestingBot runs both on the same grid, in parallel, in one dashboard. Background ## What are Cypress and Playwright? Two modern JavaScript testing frameworks with different architectures. The same goal: drive a browser to automate end-to-end testing. ![Cypress](https://testingbot.com/assets/support/cypress-409c9fd3f79115ed7676aa1df4f6a217dc4402b3366a61984924aec7b4fefa97.svg) ### Cypress Released 2017 · Cypress.io · MIT Cypress is a JavaScript end-to-end testing framework that runs your tests inside the browser, in the same run loop as your application. That architecture gives it direct DOM access, automatic retries and time-travel snapshots out of the box. Cypress is known for its developer experience: a built-in test runner, real-time reloads, a visual debugger and strong component testing. The in-browser model also limits it to JavaScript and TypeScript, a fixed set of Chromium-based browsers plus Firefox, and single-tab flows. - JavaScript and TypeScript only - Auto-retries, time-travel debugging, component testing - Chrome, Edge, Firefox, Electron · no WebKit/Safari ![Playwright](https://testingbot.com/assets/support/playwright-2dcb4e75b55ac0d1135cfa47be6137b13f46a2047ce78b96a34f51b0e2cc0d96.svg) ### Playwright Released 2020 · Microsoft · Apache 2.0 Playwright is an end-to-end testing framework built by Microsoft that controls Chromium, Firefox and WebKit through a single API. It drives the browser out-of-process over the DevTools protocol (and its own bridges for Firefox and WebKit). Playwright was designed for scale: bindings in five languages, native multi-tab and multi-origin support through browser contexts, built-in parallel workers, network interception and a rich Trace Viewer, all included. - TypeScript, JavaScript, Python, Java, C# - Auto-wait, browser contexts, Trace Viewer, free parallel - Chromium, Firefox, WebKit · no IE / legacy Safari Head to head ## Cypress vs Playwright: side-by-side comparison Across the dimensions that matter for picking, migrating or running both in CI. Cypress versus Playwright feature comparison| Dimension | Cypress | Playwright | | --- | --- | --- | | First release | 2017 | 2020 | | Maintained by | Cypress.io | Microsoft | | Languages | JavaScript / TypeScript only | TS, JS, Python, Java, C# | | Architecture | Runs inside the browser | Drives browser out-of-process | | Browsers | Chrome, Edge, Firefox, Electron | Chromium, Firefox, WebKit | | WebKit / Safari engine | Experimental | Supported | | Auto-waiting | Built-in retries | Built-in auto-wait | | Multiple tabs / windows | Not supported | Native (browser contexts) | | Cross-origin | cy.origin() (constrained) | Native | | Network interception | First-class (cy.intercept) | First-class (route) | | Test runner | Built in | Built in (@playwright/test) | | Component testing | First-class | Experimental | | Parallel execution | Cypress Cloud (paid) or a grid | Built-in workers (free) | | Debugging | Time-travel runner | Trace Viewer + UI mode | | Mobile testing | Viewport emulation only | Viewport emulation only | | Ecosystem maturity | Since 2017 | Since 2020, growing fast | | Free for open source on TestingBot | ✓ | ✓ | Feature notes reflect Cypress 13.x and Playwright 1.x as of 2026. Both frameworks run on TestingBot's cloud, Cypress through the testingbot-cypress-cli and Playwright through the wsEndpoint connection. The same test in both ## Logging in, asserting the result A login flow with built-in retries (Cypress) and auto-wait (Playwright). Both run on the same TestingBot grid. Cypress + TypeScript login.cy.ts ```typescript // run via: testingbot-cypress run describe('login', () => { it('redirects to the dashboard', () => { cy.visit('https://app.example.com/login'); // auto-retries until actionable cy.get('#username').type('jane@example.com'); cy.get('#password').type('••••••••'); cy.contains('button', 'Sign in').click(); cy.url().should('include', '/dashboard'); cy.contains('h1', /welcome, jane/i).should('be.visible'); }); }); ``` Playwright + TypeScript login.spec.ts ```typescript // playwright.config.ts → set TestingBot wsEndpoint import { test, expect } from '@playwright/test'; test('login redirects to dashboard', async ({ page }) => { await page.goto('https://app.example.com/login'); // auto-wait, no explicit wait needed await page.getByLabel('Username').fill('jane@example.com'); await page.getByLabel('Password').fill('••••••••'); await page.getByRole('button', { name: 'Sign in' }).click(); await expect(page).toHaveURL(/\/dashboard/); }); ``` The Cypress spec runs through the [testingbot-cypress-cli](https://testingbot.com/support/web-automate/cypress); the Playwright test connects to the [wsEndpoint](https://testingbot.com/support/web-automate/playwright). Both appear in the same TestingBot dashboard. Decision matrix ## When to choose which ### Choose Cypress when - You want the best in-class developer experience: in-browser runner, time-travel snapshots and readable failures. - You do component testing alongside end-to-end tests and want one tool for both. - Your targets are Chrome, Edge and Firefox, with no WebKit or Safari-engine requirement. - Your team is fully JavaScript or TypeScript and values a gentle learning curve. - Your flows stay within a single tab and mostly a single origin (cy.origin helps, within limits). [Run Cypress on TestingBot](https://testingbot.com/features/automation/cypress) ### Choose Playwright when - You need WebKit coverage to approximate Safari, which Cypress cannot drive. - Your team writes Python, Java or C# as well as JavaScript and wants one framework across all of them. - Your tests open multiple tabs, windows or origins, which Playwright handles natively through browser contexts. - You want free, built-in parallel execution without a paid dashboard subscription. - You want the fastest CI runs and trace-based debugging with the Playwright Trace Viewer. [Run Playwright on TestingBot](https://testingbot.com/features/automation/playwright) TestingBot supports both ## Stop choosing, run both on the same grid Run Cypress through the testingbot-cypress-cli and connect Playwright through its wsEndpoint. Your tests share the same 6100+ browsers and devices, the same dashboard, the same parallel slots and the same EU data residency, with no Cypress Cloud subscription required. - Cypress parallelisation without a Cypress Cloud plan - Side-by-side test history for both frameworks - Free for open source, both frameworks [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) testingbot.com / run both // Cypress $ npm i -D testingbot-cypress-cli $ testingbot-cypress run # reads testingbot.json // Playwright use: { wsEndpoint: 'wss://cloud.testingbot.com/playwright' } FAQ ## Frequently Asked Questions The questions teams ask before picking, or migrating between, these frameworks. Is Playwright better than Cypress? Neither is universally better; they fit different teams. Playwright is more flexible: it drives Chromium, Firefox and WebKit, supports five languages, handles multiple tabs and origins natively, and parallelises for free. Cypress is loved for its developer experience: an in-browser runner, time-travel debugging and first-class component testing. Pick Playwright for breadth and scale; pick Cypress for ergonomics within its constraints. Should I migrate from Cypress to Playwright? Only if you are hitting Cypress's limits. If you need WebKit (Safari) coverage, more than one language, multi-tab or multi-origin flows, or free parallel execution, Playwright removes those constraints. If your Cypress suite is stable, JavaScript-only and within a single tab, the migration is rarely worth the effort. Many teams keep their Cypress suite and add Playwright for new cross-browser work. Does Cypress or Playwright support more browsers? Playwright supports more rendering engines. It drives Chromium (Chrome, Edge), Firefox and WebKit, which approximates Safari. Cypress runs on Chromium-based browsers (Chrome, Edge, Electron) and Firefox, with experimental WebKit support. Neither drives the real Safari binary or Internet Explorer; for those you need [Selenium](https://testingbot.com/features/automation/selenium). TestingBot runs both frameworks across its browser grid. Which is faster, Cypress or Playwright? Playwright is usually faster for full suites, mainly because it parallelises across workers for free out of the box and drives the browser out-of-process. Cypress runs each spec in a single browser instance and needs Cypress Cloud or a cloud grid to parallelise. On a single spec the difference is small; across a large suite, parallelisation is the deciding factor. Running either on TestingBot's grid parallelises across browsers regardless. Can Cypress or Playwright test mobile apps? Neither tests native mobile apps. Both offer mobile viewport emulation in Chromium, but cannot drive native iOS or Android apps. For real mobile testing you need [Appium](https://testingbot.com/features/automation/appium), [XCUITest](https://testingbot.com/features/automation/xcuitest), [Espresso](https://testingbot.com/features/automation/espresso) or [Maestro](https://testingbot.com/features/automation/maestro). TestingBot runs all of these on real iOS and Android devices. Do I need Cypress Cloud, or can I parallelise for free? Playwright parallelises for free with its built-in worker pool. Cypress's own parallelisation and recording run through Cypress Cloud, a paid subscription. TestingBot is an alternative for Cypress: the testingbot-cypress-cli runs your specs in parallel across Chrome, Edge and Firefox on the cloud grid, with live logs and video, and no Cypress Cloud plan. You set the parallel count in testingbot.json. Can I run Cypress and Playwright on TestingBot? Yes, both run on the same TestingBot cloud. Cypress runs through the `testingbot-cypress-cli` npm package, configured with a `testingbot.json` file and started with `testingbot-cypress run`. Playwright connects via its `wsEndpoint` to `wss://cloud.testingbot.com/playwright`. Tests from both show up in the same dashboard, share parallel slots, and benefit from the same EU data residency, video recording and CI/CD integrations. Both are free for open source projects. Going deeper? See the dedicated [Cypress](https://testingbot.com/features/automation/cypress) and [Playwright](https://testingbot.com/features/automation/playwright) pages. ## Related comparisons [Cypress vs Selenium](https://testingbot.com/cypress-vs-selenium) [Playwright vs Selenium](https://testingbot.com/playwright-vs-selenium) ### Sign up for a Free Trial Run Cypress, Playwright, or both, on TestingBot's cloud. No Cypress Cloud subscription required. [Start a free trial](https://testingbot.com/users/sign_up) Resources you may like [![Tutorial on debugging Cypress tests](https://testingbot.com/assets/resources/articles/10-7ae9f6d9435bea1c54ec815ad535dc4a590086e6165263a9eba3441635d55726.webp)](https://testingbot.com/resources/articles/how-to-debug-cypress-tests) ### [Tutorial on debugging Cypress tests](https://testingbot.com/resources/articles/how-to-debug-cypress-tests) How to debug a failing Cypress test using the debugger, console logs, the stack trace, and the debug and pause commands. [Read more: Tutorial on debugging Cypress tests](https://testingbot.com/resources/articles/how-to-debug-cypress-tests) [![Visual Testing with Playwright](https://testingbot.com/assets/resources/articles/14-f3f2e8954c8a47cf0bb29059b44f4d2f40998442e22db6f8acdccb8e5b34d8c1.webp)](https://testingbot.com/resources/articles/playwright-visual-regression-testing) ### [Visual Testing with Playwright](https://testingbot.com/resources/articles/playwright-visual-regression-testing) Playwright provides automated browser testing. It offers a built-in feature to perform visual regression testing for your website. [Read more: Visual Testing with Playwright](https://testingbot.com/resources/articles/playwright-visual-regression-testing) [![Cypress and Cucumber Browser Testing](https://testingbot.com/assets/resources/articles/6-495c5e14cb4bf1e282858fc7e2cfcbe89775e3790b88d34c15bf9a6f36d387d4.webp)](https://testingbot.com/resources/articles/cypress-cucumber-testing) ### [Cypress and Cucumber Browser Testing](https://testingbot.com/resources/articles/cypress-cucumber-testing) How to combine Cypress with Cucumber so end-to-end tests read as plain language, including a worked example and the TypeScript setup. [Read more: Cypress and Cucumber Browser Testing](https://testingbot.com/resources/articles/cypress-cucumber-testing) --- URL: https://testingbot.com/selenium-vs-puppeteer Framework comparison · 2026 # Selenium vs Puppeteer Selenium and Puppeteer solve different problems: one is the cross-browser W3C WebDriver standard, the other is a fast Chrome-focused automation library. This guide compares them head-to-head on protocol, languages, browser coverage, speed and use cases, with code samples and a verdict you can act on in five minutes. [Run both on TestingBot](https://testingbot.com/users/sign_up) [Jump to comparison](https://testingbot.com#comparison) - **Browsers & devices:** 6100+ - **Cloud parallelism:** 100× - **Uptime SLA:** 99.9% ## Trusted by some of the world's most innovative companies 30-second answer ## Which one should you pick? ![Selenium](https://testingbot.com/assets/support/selenium-266ecc0599068fc6eca0bcd9174fb46053c95cebe1c66f7a50c480f3f1a52929.svg) Pick Selenium if… You need real cross-browser testing across Chrome, Firefox, Safari, Edge and Internet Explorer, more than one language, or real mobile devices through Appium, on a W3C-standard protocol. ![Puppeteer](https://testingbot.com/assets/support/puppeteer-0a8cdd38af69dabd46638afbb3bffc8a372902949aa599663ec379f3e875263c.svg) Pick Puppeteer if… You automate Chrome or Chromium from Node.js for fast end-to-end tests, scraping, PDF generation or performance tracing, and you do not need Safari, Internet Explorer or other languages. Pick both if… You use Puppeteer for fast Chrome-only checks and Selenium for broad cross-browser coverage. TestingBot runs both on the same grid, in parallel, in one dashboard. Background ## What are Selenium and Puppeteer? Two browser automation tools with different goals. One standardises cross-browser control; the other drives Chrome at speed. ![Selenium](https://testingbot.com/assets/support/selenium-266ecc0599068fc6eca0bcd9174fb46053c95cebe1c66f7a50c480f3f1a52929.svg) ### Selenium Released 2004 · Open Source · Apache 2.0 Selenium is the original browser automation framework and the foundation of the W3C WebDriver standard. Selenium WebDriver drives any browser from the outside through a vendor-supplied driver (chromedriver, geckodriver, safaridriver, edgedriver) using the same wire protocol. Twenty years of ecosystem maturity means bindings for every major language, deep IDE integration, mature Page Object patterns, and real mobile device testing through Appium, which is built on the same protocol. - Java / Python / C# / Ruby / JavaScript / Kotlin - W3C WebDriver standard, works with every browser - Chrome, Firefox, Safari, Edge, IE 11 · real mobile via Appium ![Puppeteer](https://testingbot.com/assets/support/puppeteer-0a8cdd38af69dabd46638afbb3bffc8a372902949aa599663ec379f3e875263c.svg) ### Puppeteer Released 2017 · Google · Apache 2.0 Puppeteer is a Node.js library from the Google Chrome team that controls Chrome, Chromium and Edge through the Chrome DevTools Protocol. It powers fast end-to-end testing, web scraping, PDF generation and performance tracing. Because it talks the DevTools protocol directly instead of WebDriver, Puppeteer is fast and has first-class access to the network, the console and tracing. The same design ties it to Chromium-based browsers and the JavaScript ecosystem. - JavaScript and TypeScript (Node.js) - Chrome DevTools Protocol, network and tracing access - Chrome, Chromium, Edge · Firefox experimental, no Safari/IE Head to head ## Selenium vs Puppeteer: side-by-side comparison Across the dimensions that matter for picking, migrating or running both in CI. Selenium versus Puppeteer feature comparison| Dimension | Selenium | Puppeteer | | --- | --- | --- | | First release | 2004 | 2017 | | Maintained by | Open-source community + W3C | Google (Chrome team) | | Protocol | W3C WebDriver | Chrome DevTools Protocol | | Languages | Java, Python, C#, Ruby, JS, Kotlin | JavaScript / TypeScript | | Browsers | Chrome, Firefox, Safari, Edge, IE 11 | Chrome, Chromium, Edge | | Firefox / Safari | Both supported | Firefox experimental, no Safari | | Mobile testing | Real iOS + Android via Appium | None | | Speed | Mature, predictable | Faster on Chrome (direct CDP) | | Network interception | BiDi (Selenium 4) or proxy | First-class (CDP) | | Scraping / PDF / tracing | Possible | First-class | | Test runner | Bring your own (JUnit, pytest) | Bring your own (Jest, Mocha) | | Standard | W3C WebDriver standard | Chrome-specific protocol | | Parallel execution | Selenium Grid | Worker-based (Jest, Mocha) | | Ecosystem maturity | 20 years of integrations | Strong since 2017 | | Free for open source on TestingBot | ✓ | ✓ | Feature notes reflect Selenium 4.x and Puppeteer 23.x as of 2026. Both run on TestingBot's cloud, Selenium through the WebDriver hub and Puppeteer through the browserWSEndpoint connection. The same test in both ## Logging in, asserting the result A login flow with an explicit wait (Selenium) and a DevTools-protocol session (Puppeteer). Both run on the same TestingBot grid. Selenium + Python test\_login.py ```python # Driver points at TestingBot remote URL from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC def test_login_redirects_to_dashboard(driver): driver.get('https://app.example.com/login') driver.find_element(By.NAME, 'username').send_keys('jane@example.com') driver.find_element(By.NAME, 'password').send_keys('••••••••') driver.find_element(By.CSS_SELECTOR, 'button[type=submit]').click() # explicit wait WebDriverWait(driver, 10).until(EC.url_contains('/dashboard')) welcome = driver.find_element(By.TAG_NAME, 'h1') assert 'Welcome, Jane' in welcome.text ``` Puppeteer + Node.js login.test.js ```javascript // connect to TestingBot over the DevTools protocol import puppeteer from 'puppeteer-core'; const caps = { key: 'KEY', secret: 'SECRET', browserName: 'chrome', browserVersion: 'latest' }; const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(caps))}`, }); const page = await browser.newPage(); await page.goto('https://app.example.com/login'); await page.type('#username', 'jane@example.com'); await page.type('#password', '••••••••'); await page.click('button[type=submit]'); await page.waitForSelector('h1'); ``` The Selenium test connects to the [WebDriver hub](https://testingbot.com/support/web-automate/selenium); the Puppeteer script connects via [browserWSEndpoint](https://testingbot.com/support/web-automate/puppeteer). Both appear in the same TestingBot dashboard. Decision matrix ## When to choose which ### Choose Selenium when - You need true cross-browser coverage, including Safari, Internet Explorer 11 and Edge, not just Chromium. - Your team writes Java, C#, Ruby or Python and wants full client-library parity. - You test real iOS and Android devices, where Appium reuses the same WebDriver protocol. - You want a vendor-neutral, W3C-standard protocol rather than a Chrome-specific one. - You have a mature Page Object library or BDD framework (Cucumber, SpecFlow) you do not want to rewrite. [Run Selenium on TestingBot](https://testingbot.com/features/automation/selenium) ### Choose Puppeteer when - You only need to automate Chrome, Chromium or Edge and want the fastest possible runs. - Your team is Node.js-first and you want a JavaScript or TypeScript API. - You do web scraping, PDF generation, screenshots or performance tracing alongside testing. - You want direct DevTools-protocol access to the network, console and CDP events. - You do not need Safari, Internet Explorer, real mobile devices or non-JavaScript languages. [Run Puppeteer on TestingBot](https://testingbot.com/features/automation/puppeteer) TestingBot supports both ## Stop choosing, run both on the same grid Point Selenium at the WebDriver hub and connect Puppeteer through its browserWSEndpoint. Your tests share the same 6100+ browsers and devices, the same dashboard, the same parallel slots and the same EU data residency. - Same auth, same project, same billing - Side-by-side test history for both tools - Free for open source, both tools [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) testingbot.com / run both # Selenium webdriver.Remote( command\_executor='https://hub.testingbot.com/wd/hub' ) // Puppeteer puppeteer.connect({ browserWSEndpoint: 'wss://cloud.testingbot.com/puppeteer' }) FAQ ## Frequently Asked Questions The questions teams ask before picking, or combining, these tools. Is Puppeteer faster than Selenium? Usually yes, on Chrome. Puppeteer talks the Chrome DevTools Protocol directly, avoiding the HTTP round-trips of the WebDriver protocol, so Chrome-only suites often run faster. But the gain only applies to Chromium-based browsers, and it narrows once tests are network-bound. Selenium 4's BiDi support also closes the gap. Speed is real but rarely the deciding factor against Selenium's cross-browser reach. Can Puppeteer replace Selenium? Only for Chrome-focused work. Puppeteer is excellent for automating Chrome, Chromium and Edge from Node.js, but it cannot drive Safari or Internet Explorer, has no real mobile support, and is JavaScript-only. If you need cross-browser testing, multiple languages or real devices, Selenium remains the broader tool. Many teams use Puppeteer for fast Chrome checks and Selenium for full cross-browser coverage. Does Puppeteer support browsers other than Chrome? Puppeteer drives Chrome, Chromium and Chromium-based Edge, with experimental Firefox support. It cannot drive Safari or Internet Explorer. Selenium drives the real binaries of every major browser through vendor drivers, including Safari and Internet Explorer 11. If you need anything beyond Chromium, Selenium is the option of the two. TestingBot runs both across its browser grid. Can Puppeteer test mobile apps? No. Puppeteer offers mobile viewport emulation in Chromium, but it cannot drive native iOS or Android apps. For real mobile testing you need [Appium](https://testingbot.com/features/automation/appium), [XCUITest](https://testingbot.com/features/automation/xcuitest), [Espresso](https://testingbot.com/features/automation/espresso) or [Maestro](https://testingbot.com/features/automation/maestro). Appium reuses the WebDriver protocol Selenium is built on. TestingBot runs all of these on real iOS and Android devices. Is Puppeteer or Selenium better for web scraping? Puppeteer is the more common choice for scraping, PDF generation and headless Chrome automation, thanks to its direct DevTools access and Node.js ecosystem. Selenium can scrape too, but its strength is cross-browser test automation across many languages. Pick the tool that matches the job: Puppeteer for Chrome automation tasks, Selenium for standards-based cross-browser testing. Can I run Puppeteer in parallel on TestingBot? Yes. Puppeteer's worker-based parallelism (via Jest, Mocha or a custom runner) works against the TestingBot grid: each worker opens its own remote browser over the WebSocket endpoint. Concurrency is capped by your plan. Selenium parallelises the same way through the WebDriver hub. Can I run Selenium and Puppeteer on TestingBot? Yes, both run on the same TestingBot cloud. Selenium connects to `https://hub.testingbot.com/wd/hub` via `webdriver.Remote`. Puppeteer connects with `puppeteer.connect` using a `browserWSEndpoint` of `wss://cloud.testingbot.com/puppeteer`. Tests from both show up in the same dashboard, share parallel slots, and benefit from the same EU data residency, video recording and CI/CD integrations. Both are free for open source projects. Going deeper? See the dedicated [Selenium](https://testingbot.com/features/automation/selenium) and [Puppeteer](https://testingbot.com/features/automation/puppeteer) pages. ## Related comparisons [Playwright vs Selenium](https://testingbot.com/playwright-vs-selenium) [Cypress vs Playwright](https://testingbot.com/cypress-vs-playwright) ### Sign up for a Free Trial Run Selenium, Puppeteer, or both, on TestingBot's cloud. [Start a free trial](https://testingbot.com/users/sign_up) Resources you may like [![Selenium and Generative AI](https://testingbot.com/assets/resources/articles/28-af707bf052844eb00f861bb6ebba57e6be4479dde1c78a5df4e70d2f597a8b18.webp)](https://testingbot.com/resources/articles/generative-ai-selenium) ### [Selenium and Generative AI](https://testingbot.com/resources/articles/generative-ai-selenium) How to use generative AI to produce realistic test data for Selenium runs, and why varied data uncovers bugs that fixed fixtures never will. [Read more: Selenium and Generative AI](https://testingbot.com/resources/articles/generative-ai-selenium) [![Automated Testing with Puppeteer](https://testingbot.com/assets/resources/articles/11-416015eb394d928299663409dccd67993b0a664f4b3ff456803fefc6bf1f325a.webp)](https://testingbot.com/resources/articles/automated-testing-with-puppeteer) ### [Automated Testing with Puppeteer](https://testingbot.com/resources/articles/automated-testing-with-puppeteer) Puppeteer combined with a test framework provides a great way to run automated browser tests. Follow this guide for more information. [Read more: Automated Testing with Puppeteer](https://testingbot.com/resources/articles/automated-testing-with-puppeteer) [![Visual Testing with Playwright](https://testingbot.com/assets/resources/articles/14-f3f2e8954c8a47cf0bb29059b44f4d2f40998442e22db6f8acdccb8e5b34d8c1.webp)](https://testingbot.com/resources/articles/playwright-visual-regression-testing) ### [Visual Testing with Playwright](https://testingbot.com/resources/articles/playwright-visual-regression-testing) Playwright provides automated browser testing. It offers a built-in feature to perform visual regression testing for your website. [Read more: Visual Testing with Playwright](https://testingbot.com/resources/articles/playwright-visual-regression-testing) --- URL: https://testingbot.com/appium-vs-espresso Framework comparison · 2026 # Appium vs Espresso Appium and Espresso are the two leading ways to automate Android UI tests. Appium is a cross-platform, black-box framework on the W3C WebDriver protocol; Espresso is Google's fast, white-box framework for Android only. This guide compares them head-to-head, with code samples and a verdict you can act on in five minutes. [Run both on TestingBot](https://testingbot.com/users/sign_up) [Jump to comparison](https://testingbot.com#comparison) - **Browsers & devices:** 6100+ - **Cloud parallelism:** 100× - **Uptime SLA:** 99.9% ## Trusted by some of the world's most innovative companies 30-second answer ## Which one should you pick? ![Appium](https://testingbot.com/assets/support/appium-4b35a8bb56bae236bd84c539cee9b8bbb489a55ddf48e32d059c86685d230354.svg) Pick Appium if… You want one test suite that runs on both Android and iOS, your team works in Java, Python, JavaScript, Ruby or C#, or you test apps you cannot rebuild with instrumentation. ![Espresso](https://testingbot.com/assets/integrations/espresso-a8ff30da0c385f89f4e694bb29fc4779ad132e8e3f4a777a38e608e0492ff610.svg) Pick Espresso if… You test Android only, you own the app source, and you want the fastest, least flaky UI tests with automatic synchronisation, written in Kotlin or Java alongside your app. Pick both if… You use Espresso for fast Android-only component and UI tests and Appium for cross-platform end-to-end coverage. TestingBot runs both on the same real-device grid, in parallel. Background ## What are Appium and Espresso? Two mobile UI automation frameworks with opposite designs. One is cross-platform and black-box; the other is Android-native and white-box. ![Appium](https://testingbot.com/assets/support/appium-4b35a8bb56bae236bd84c539cee9b8bbb489a55ddf48e32d059c86685d230354.svg) ### Appium Released 2012 · OpenJS Foundation · Apache 2.0 Appium is an open-source, cross-platform test framework for native, hybrid and mobile-web apps. It implements the W3C WebDriver protocol, so the same client library drives both iOS and Android from Java, Python, Node.js, Ruby, C# and more. Appium drives the app from the outside as a black box, so it needs no app source code. On Android it uses the UiAutomator2 or Espresso driver under the hood; on iOS it uses XCUITest. That layer is what makes one test reusable across platforms. - Java / Python / Node.js / Ruby / C# / PHP - Cross-platform, native, hybrid and mobile web - Black-box, no app source required ![Espresso](https://testingbot.com/assets/integrations/espresso-a8ff30da0c385f89f4e694bb29fc4779ad132e8e3f4a777a38e608e0492ff610.svg) ### Espresso Released 2013 · Google · Apache 2.0 Espresso is Google's UI testing framework for Android, part of the Android Jetpack test libraries. Tests run in-process on the device, in the same process as the app under test, which gives them direct access to the app and automatic synchronisation with the UI thread. Because Espresso runs inside the app process, it is fast and reliable, with very little flakiness. The trade-offs: it is Android only, it is written in Kotlin or Java, and it needs an instrumented build of the app. - Kotlin and Java - In-process, automatic UI synchronisation - Android only · needs an instrumented build Head to head ## Appium vs Espresso: side-by-side comparison Across the dimensions that matter for Android UI testing and cross-platform coverage. Appium versus Espresso feature comparison| Dimension | Appium | Espresso | | --- | --- | --- | | First release | 2012 | 2013 | | Maintained by | Open source (OpenJS Foundation) | Google | | Platforms | iOS + Android | Android only | | Languages | Java, Python, JS, Ruby, C#, PHP | Java, Kotlin | | Protocol | W3C WebDriver | Android instrumentation | | Test execution | Out-of-process (black-box) | In-process (white-box) | | Speed | Slower (WebDriver round-trips) | Very fast (in-process) | | App source needed | No (black-box) | Yes (instrumented build) | | UI synchronization | Manual waits | Automatic, built in | | Cross-platform reuse | Same test on iOS + Android | Android only | | Web / hybrid apps | Yes | Limited | | Flakiness | Good | Excellent (tight coupling) | | On TestingBot | Upload app, WebDriver hub | Upload app + test, CLI | | Free for open source on TestingBot | ✓ | ✓ | Both run on TestingBot's real Android devices and emulators. Appium connects to the WebDriver hub with an uploaded app; Espresso runs through the TestingBot CLI. The same login test ## Logging in on Android A login flow driven black-box by Appium and white-box by Espresso. Both run on the same real Android devices on TestingBot. Appium + Python test\_login.py ```python # real Android device on TestingBot from appium import webdriver from appium.options.android import UiAutomator2Options from appium.webdriver.common.appiumby import AppiumBy opts = UiAutomator2Options().load_capabilities({ 'platformName': 'Android', 'appium:app': 'tb://APP_ID', 'tb:options': { 'realDevice': True }, }) d = webdriver.Remote('https://hub.testingbot.com/wd/hub', options=opts) d.find_element(AppiumBy.ACCESSIBILITY_ID, 'username').send_keys('jane') d.find_element(AppiumBy.ACCESSIBILITY_ID, 'sign_in').click() assert d.find_element(AppiumBy.ID, 'welcome').is_displayed() ``` Espresso + Kotlin LoginTest.kt ```kotlin // runs via the TestingBot CLI @Test fun loginRedirectsToDashboard() { onView(withId(R.id.username)) .perform(typeText("jane@example.com")) onView(withId(R.id.password)) .perform(typeText("secret")) onView(withId(R.id.sign_in)) .perform(click()) onView(withId(R.id.welcome)) .check(matches(isDisplayed())) } ``` The Appium test connects to the [WebDriver hub](https://testingbot.com/support/app-automate/appium) with an uploaded app; the Espresso suite runs through the [TestingBot CLI](https://testingbot.com/support/app-automate/espresso). Both run on real Android devices. Decision matrix ## When to choose which ### Choose Appium when - You need one test suite that runs on both Android and iOS without a rewrite. - Your team writes Java, Python, JavaScript, Ruby or C# rather than Kotlin. - You test an app you cannot rebuild with instrumentation, or a third-party or hybrid app. - You already use Selenium or WebDriver and want the same protocol for mobile. - You test native, hybrid and mobile-web flows with one tool. [Run Appium on TestingBot](https://testingbot.com/features/automation/appium) ### Choose Espresso when - You test Android only and own the app source, so an instrumented build is fine. - You want the fastest, least flaky UI tests with automatic synchronisation. - Your team writes Kotlin or Java and wants tests next to the app code. - You test internal app state, custom views or components, not just the public UI. - You run UI tests as part of the Android build and CI from day one. [Run Espresso on TestingBot](https://testingbot.com/features/automation/espresso) TestingBot supports both ## Stop choosing, run both on the same grid Upload your app and drive it black-box with Appium over the WebDriver hub, or run your Espresso suite through the TestingBot CLI. Both run on the same real Android devices, with the same dashboard, parallel slots, video recording and EU data residency. - Real Android devices and emulators - Side-by-side test history for both frameworks - Free for open source, both frameworks [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) testingbot.com / run both # Appium appium:app = 'tb://APP\_ID' hub: https://hub.testingbot.com/wd/hub # Espresso (@testingbot/cli) $ testingbot espresso app.apk app-test.apk --device "Galaxy S24" --real-device FAQ ## Frequently Asked Questions The questions teams ask before picking, or combining, these frameworks. Is Espresso faster than Appium? Yes, on Android. Espresso runs in-process, in the same process as the app, so it avoids the network round-trips of Appium's WebDriver protocol and synchronises automatically with the UI thread. That makes Espresso faster and less flaky. The trade-off is that Espresso is Android only and needs an instrumented build, while Appium runs the same test across Android and iOS as a black box. Should I use Appium or Espresso for Android? It depends on your goals. For Android-only UI tests where you own the source, Espresso gives the fastest, most reliable runs. For cross-platform coverage, tests in a non-Kotlin language, or apps you cannot instrument, Appium is the better fit. Many Android teams run Espresso for component and UI tests and Appium for cross-platform end-to-end flows. Can Espresso test iOS apps? No. Espresso is an Android-only framework from Google. To test iOS you need [XCUITest](https://testingbot.com/features/automation/xcuitest) (Apple's native framework) or [Appium](https://testingbot.com/features/automation/appium), which drives iOS through XCUITest under the hood. Appium is the only one of these that runs the same test on both platforms. TestingBot runs all of them on real devices. Can I reuse the same tests across Android and iOS? With Appium, yes. Because Appium implements the W3C WebDriver protocol and abstracts the native drivers (UiAutomator2 or Espresso on Android, XCUITest on iOS), a single test can target both platforms with minor capability changes. Espresso cannot do this; it is Android only. If cross-platform reuse matters, Appium is the framework to choose. Does Espresso need the app source code? Yes. Espresso compiles into an instrumented test APK that runs alongside your app, so you need the app source or at least a debuggable, instrumented build. Appium drives the installed app as a black box and needs no source, which is why it can test third-party or already-published apps that Espresso cannot. Can I run Appium and Espresso on real devices in the cloud? Yes. TestingBot provides real Android phones and tablets, plus emulators, in an EU datacenter. Appium tests upload your .apk to TestingBot Storage and run against the hub with realDevice set; Espresso tests upload the app and test APKs through the TestingBot CLI. Both record video and logs you can review in the dashboard. Can I run Appium and Espresso on TestingBot? Yes, both run on the same TestingBot grid. Appium uploads your app to TestingBot Storage, then connects to `https://hub.testingbot.com/wd/hub` with an `appium:app` capability of `tb://APP_ID`. Espresso runs through the TestingBot CLI: `testingbot espresso app.apk app-test.apk --device "Galaxy S24" --real-device`. Both share the same dashboard, parallel slots and EU data residency, and are free for open source. Going deeper? See the dedicated [Appium](https://testingbot.com/features/automation/appium) and [Espresso](https://testingbot.com/features/automation/espresso) pages. ## Related comparisons [Appium vs XCUITest](https://testingbot.com/appium-vs-xcuitest) [Playwright vs Selenium](https://testingbot.com/playwright-vs-selenium) ### Sign up for a Free Trial Run Appium and Espresso on real Android devices in TestingBot's cloud. [Start a free trial](https://testingbot.com/users/sign_up) Resources you may like [![Using Touch Actions with Appium](https://testingbot.com/assets/resources/articles/24-490f70ad8a80e4534523707125b88e44e70fe19b126be4f44e689bbd5f6a0cbf.webp)](https://testingbot.com/resources/articles/touch-actions-appium) ### [Using Touch Actions with Appium](https://testingbot.com/resources/articles/touch-actions-appium) How to simulate taps, swipes and pinches on real devices with Appium Touch Actions, with examples, best practices and common pitfalls. [Read more: Using Touch Actions with Appium](https://testingbot.com/resources/articles/touch-actions-appium) [![Android Espresso Tutorial](https://testingbot.com/assets/resources/articles/19-fe4971eb8049097c25f2cc2fce95b71b94f3bb342e094c219b2404c09374f015.webp)](https://testingbot.com/resources/articles/android-espresso-testing) ### [Android Espresso Tutorial](https://testingbot.com/resources/articles/android-espresso-testing) Why Espresso suits Android UI testing, how to add it to a project and write a first test, and how it compares with the alternatives. [Read more: Android Espresso Tutorial](https://testingbot.com/resources/articles/android-espresso-testing) [![Appium 2 and Appium 3 Migration Guide](https://testingbot.com/assets/resources/articles/27-121da26036885301ba438e1cdfc81e7ae8045cbc53c0dafaff5b4436000590b1.webp)](https://testingbot.com/resources/articles/appium-2-migration) ### [Appium 2 and Appium 3 Migration Guide](https://testingbot.com/resources/articles/appium-2-migration) What changed between Appium 1, 2 and 3: drivers, capabilities, the removed touch endpoints, and how to upgrade an existing suite. [Read more: Appium 2 and Appium 3 Migration Guide](https://testingbot.com/resources/articles/appium-2-migration) --- URL: https://testingbot.com/appium-vs-xcuitest Framework comparison · 2026 # Appium vs XCUITest Appium and XCUITest are the two leading ways to automate iOS UI tests. Appium is a cross-platform, black-box framework on the W3C WebDriver protocol; XCUITest is Apple's fast, native framework for iOS only. This guide compares them head-to-head, with code samples and a verdict you can act on in five minutes. [Run both on TestingBot](https://testingbot.com/users/sign_up) [Jump to comparison](https://testingbot.com#comparison) - **Browsers & devices:** 6100+ - **Cloud parallelism:** 100× - **Uptime SLA:** 99.9% ## Trusted by some of the world's most innovative companies 30-second answer ## Which one should you pick? ![Appium](https://testingbot.com/assets/support/appium-4b35a8bb56bae236bd84c539cee9b8bbb489a55ddf48e32d059c86685d230354.svg) Pick Appium if… You want one test suite that runs on both iOS and Android, your team works in Java, Python, JavaScript, Ruby or C#, or you test apps you cannot rebuild with a test target. ![XCUITest](https://testingbot.com/assets/integrations/xcuitest-4a935f2368661d89d3559b3c5af3377236029478ae76bb43253f3c3bee712096.svg) Pick XCUITest if… You test iOS only, you own the app in Xcode, and you want the fastest, most reliable native UI tests written in Swift or Objective-C alongside your app. Pick both if… You use XCUITest for fast iOS-only native tests and Appium for cross-platform end-to-end coverage. TestingBot runs both on the same real-device grid, in parallel. Background ## What are Appium and XCUITest? Two iOS UI automation frameworks with opposite designs. One is cross-platform and black-box; the other is Apple-native and tightly coupled to Xcode. ![Appium](https://testingbot.com/assets/support/appium-4b35a8bb56bae236bd84c539cee9b8bbb489a55ddf48e32d059c86685d230354.svg) ### Appium Released 2012 · OpenJS Foundation · Apache 2.0 Appium is an open-source, cross-platform test framework for native, hybrid and mobile-web apps. It implements the W3C WebDriver protocol, so the same client library drives both iOS and Android from Java, Python, Node.js, Ruby, C# and more. Appium drives the app from the outside as a black box, so it needs no app source code. On iOS it uses Apple's XCUITest under the hood (through WebDriverAgent); on Android it uses UiAutomator2. That layer is what makes one test reusable across platforms. - Java / Python / Node.js / Ruby / C# / PHP - Cross-platform, native, hybrid and mobile web - Black-box, no app source required ![XCUITest](https://testingbot.com/assets/integrations/xcuitest-4a935f2368661d89d3559b3c5af3377236029478ae76bb43253f3c3bee712096.svg) ### XCUITest Released 2015 · Apple · part of Xcode XCUITest is Apple's UI testing framework, part of the larger XCTest framework that ships with Xcode. Tests are written in Swift or Objective-C and run on-device through XCTest, with first-class access to the iOS accessibility tree. Because XCUITest is native and maintained by Apple, it is fast, reliable and always current with the latest iOS. The trade-offs: it is iOS only, it is written in Swift or Objective-C, and it needs an Xcode test target. - Swift and Objective-C - Native XCTest, accessibility-tree access - iOS only · needs an Xcode test target Head to head ## Appium vs XCUITest: side-by-side comparison Across the dimensions that matter for iOS UI testing and cross-platform coverage. Appium versus XCUITest feature comparison| Dimension | Appium | XCUITest | | --- | --- | --- | | First release | 2012 | 2015 | | Maintained by | Open source (OpenJS Foundation) | Apple | | Platforms | iOS + Android | iOS only | | Languages | Java, Python, JS, Ruby, C#, PHP | Swift, Objective-C | | Protocol | W3C WebDriver | XCTest / XCUITest (native) | | iOS engine | XCUITest via WebDriverAgent | XCUITest (direct) | | Speed | Slower (extra layers) | Faster (native) | | App source needed | No (black-box) | Yes (Xcode test target) | | Tooling | Any IDE / CI | Xcode | | Cross-platform reuse | Same test on iOS + Android | iOS only | | Web / hybrid apps | Yes | Limited | | Flakiness | Good | Excellent (native sync) | | On TestingBot | Upload app, WebDriver hub | Upload app + test zip, CLI | | Free for open source on TestingBot | ✓ | ✓ | Both run on TestingBot's real iPhones and iPads. Appium connects to the WebDriver hub with an uploaded app; XCUITest runs through the TestingBot CLI. The same login test ## Logging in on iOS A login flow driven black-box by Appium and natively by XCUITest. Both run on the same real iPhones on TestingBot. Appium + Python test\_login.py ```python # real iPhone on TestingBot from appium import webdriver from appium.options.ios import XCUITestOptions from appium.webdriver.common.appiumby import AppiumBy opts = XCUITestOptions().load_capabilities({ 'platformName': 'iOS', 'appium:app': 'tb://APP_ID', 'tb:options': { 'realDevice': True }, }) d = webdriver.Remote('https://hub.testingbot.com/wd/hub', options=opts) d.find_element(AppiumBy.ACCESSIBILITY_ID, 'username').send_keys('jane') d.find_element(AppiumBy.ACCESSIBILITY_ID, 'sign_in').click() assert d.find_element(AppiumBy.ACCESSIBILITY_ID, 'welcome').is_displayed() ``` XCUITest + Swift LoginTests.swift ```swift // runs via the TestingBot CLI func testLoginRedirectsToDashboard() { let app = XCUIApplication() app.launch() app.textFields["username"].typeText("jane@example.com") app.secureTextFields["password"].typeText("secret") app.buttons["sign_in"].tap() XCTAssertTrue(app.staticTexts["welcome"].exists) } ``` The Appium test connects to the [WebDriver hub](https://testingbot.com/support/app-automate/appium) with an uploaded app; the XCUITest suite runs through the [TestingBot CLI](https://testingbot.com/support/app-automate/xcuitest). Both run on real iPhones and iPads. Decision matrix ## When to choose which ### Choose Appium when - You need one test suite that runs on both iOS and Android without a rewrite. - Your team writes Java, Python, JavaScript, Ruby or C# rather than Swift. - You test an app you cannot rebuild with a test target, or a third-party or hybrid app. - You already use Selenium or WebDriver and want the same protocol for mobile. - You test native, hybrid and mobile-web flows with one tool. [Run Appium on TestingBot](https://testingbot.com/features/automation/appium) ### Choose XCUITest when - You test iOS only and own the app in Xcode, so a test target is fine. - You want the fastest, most reliable native iOS tests that track the latest iOS. - Your team writes Swift or Objective-C and wants tests next to the app code. - You need deep access to the iOS accessibility tree and native components. - You run UI tests as part of the Xcode build and CI from day one. [Run XCUITest on TestingBot](https://testingbot.com/features/automation/xcuitest) TestingBot supports both ## Stop choosing, run both on the same grid Upload your app and drive it black-box with Appium over the WebDriver hub, or run your XCUITest suite through the TestingBot CLI. Both run on the same real iPhones and iPads, with the same dashboard, parallel slots, video recording and EU data residency. - Real iPhones and iPads - Side-by-side test history for both frameworks - Free for open source, both frameworks [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) testingbot.com / run both # Appium appium:app = 'tb://APP\_ID' hub: https://hub.testingbot.com/wd/hub # XCUITest (@testingbot/cli) $ testingbot xcuitest app.ipa app-test.zip --device "iPhone 16" --real-device FAQ ## Frequently Asked Questions The questions teams ask before picking, or combining, these frameworks. Is XCUITest faster than Appium? Yes, on iOS. XCUITest is Apple's native framework and runs on-device through XCTest, so it avoids the extra WebDriver and WebDriverAgent layers Appium adds. That makes XCUITest faster and very reliable. The trade-off is that XCUITest is iOS only and written in Swift or Objective-C, while Appium runs the same test across iOS and Android in the language of your choice. Should I use Appium or XCUITest for iOS? It depends on your goals. For iOS-only tests where you own the Xcode project, XCUITest gives the fastest, most reliable native runs. For cross-platform coverage, tests in a non-Swift language, or apps you cannot rebuild, Appium is the better fit. Note that Appium drives iOS through XCUITest under the hood, so you are choosing the abstraction level, not a different engine. Does Appium use XCUITest internally? Yes. On iOS, Appium drives the app through Apple's XCUITest framework using a component called WebDriverAgent. So an Appium iOS test ultimately runs on top of XCUITest, with Appium adding the cross-platform WebDriver API on top. That is why Appium can offer the same test interface for iOS and Android while still using each platform's native automation underneath. Can XCUITest test Android apps? No. XCUITest is an iOS-only framework from Apple. To test Android you need [Espresso](https://testingbot.com/features/automation/espresso) (Google's native framework) or [Appium](https://testingbot.com/features/automation/appium), which drives Android through UiAutomator2. Appium is the only one of these that runs the same test on both platforms. TestingBot runs all of them on real devices. Can I reuse the same tests across iOS and Android? With Appium, yes. Because Appium implements the W3C WebDriver protocol and abstracts the native drivers (XCUITest on iOS, UiAutomator2 on Android), a single test can target both platforms with minor capability changes. XCUITest cannot do this; it is iOS only. If cross-platform reuse matters, Appium is the framework to choose. Do I need a Mac and Xcode for XCUITest? You need a Mac with Xcode to write and build XCUITest tests locally. To run them at scale you do not need your own Mac fleet: build the app .ipa and the XCUITest .zip once, then upload both to TestingBot and run on real iPhones and iPads in the cloud. Appium tests do not require Xcode on your side at all. Can I run Appium and XCUITest on TestingBot? Yes, both run on the same TestingBot grid. Appium uploads your app to TestingBot Storage, then connects to `https://hub.testingbot.com/wd/hub` with an `appium:app` capability of `tb://APP_ID`. XCUITest runs through the TestingBot CLI: `testingbot xcuitest app.ipa app-test.zip --device "iPhone 16" --real-device`. Both share the same dashboard, parallel slots and EU data residency, and are free for open source. Going deeper? See the dedicated [Appium](https://testingbot.com/features/automation/appium) and [XCUITest](https://testingbot.com/features/automation/xcuitest) pages. ## Related comparisons [Appium vs Espresso](https://testingbot.com/appium-vs-espresso) [Playwright vs Selenium](https://testingbot.com/playwright-vs-selenium) ### Sign up for a Free Trial Run Appium and XCUITest on real iPhones and iPads in TestingBot's cloud. [Start a free trial](https://testingbot.com/users/sign_up) Resources you may like [![Using Touch Actions with Appium](https://testingbot.com/assets/resources/articles/24-490f70ad8a80e4534523707125b88e44e70fe19b126be4f44e689bbd5f6a0cbf.webp)](https://testingbot.com/resources/articles/touch-actions-appium) ### [Using Touch Actions with Appium](https://testingbot.com/resources/articles/touch-actions-appium) How to simulate taps, swipes and pinches on real devices with Appium Touch Actions, with examples, best practices and common pitfalls. [Read more: Using Touch Actions with Appium](https://testingbot.com/resources/articles/touch-actions-appium) [![Automate native iOS Apps with XCUITest](https://testingbot.com/assets/resources/articles/25-bb2ccb531384c461ef792214839e9e8f99a4b08b1293730ceeab67e4a8352563.webp)](https://testingbot.com/resources/articles/automate-native-ios-apps-xcuitest) ### [Automate native iOS Apps with XCUITest](https://testingbot.com/resources/articles/automate-native-ios-apps-xcuitest) How to automate Apple's own iOS apps with XCUITest, with code for driving Settings, Messages and Photos, and how to extend it to other apps. [Read more: Automate native iOS Apps with XCUITest](https://testingbot.com/resources/articles/automate-native-ios-apps-xcuitest) [![Appium 2 and Appium 3 Migration Guide](https://testingbot.com/assets/resources/articles/27-121da26036885301ba438e1cdfc81e7ae8045cbc53c0dafaff5b4436000590b1.webp)](https://testingbot.com/resources/articles/appium-2-migration) ### [Appium 2 and Appium 3 Migration Guide](https://testingbot.com/resources/articles/appium-2-migration) What changed between Appium 1, 2 and 3: drivers, capabilities, the removed touch endpoints, and how to upgrade an existing suite. [Read more: Appium 2 and Appium 3 Migration Guide](https://testingbot.com/resources/articles/appium-2-migration) --- URL: https://testingbot.com/pricing # TestingBot Pricing TestingBot offers four cross-browser and mobile testing plans (Live, Automated, Pro, Enterprise) plus a flexible Pay-As-You-Go option. All plans include access to 6100+ real browsers and devices, video recordings, screenshots, local testing, and 24/7 support. Free plan with 100 testing minutes, no credit card required. Full-access 7-day trial with a card. All amounts on this page are in USD. Prices exclude VAT. EU customers pay 21% VAT unless they enter a valid VAT number from another EU country at checkout. Customers outside the EU pay no VAT. Subscription prices are shown both as the monthly equivalent of an annual plan and as the month-to-month rate. A parallel test (also called a concurrent session) is one test running at a time, so a 4-parallel plan runs 4 tests simultaneously. ## Live Manual testing of websites and apps. Seats are the unit here, not minutes: live testing is unlimited on every tier. | Users | Annual plan (per month) | Month to month | | ----- | ----------------------- | -------------- | | 1 | $20 | $30 | | 2 | $40 | $60 | | 4 | $80 | $120 | | 10 | $200 | $300 | | 25 | $500 | $750 | Includes unlimited live testing, unlimited visual testing, real iOS and Android devices, and 24x7 support by email, chat and Slack. ## Automated Automated web and app testing. Priced per parallel test, with a monthly allowance of automated testing minutes. | Parallel tests | Automated minutes / month | Annual plan (per month) | Month to month | | -------------- | ------------------------- | ----------------------- | -------------- | | 1 | 1,000 | $50 | $70 | | 2 | 2,000 | $120 | $140 | | 4 | 4,000 | $240 | $280 | | 8 | 6,000 | $350 | $560 | | 16 | 10,000 | $700 | $1120 | | 24 | 20,000 | $1200 | $1680 | Live and visual testing are unlimited on every tier; only automated testing draws down the minute allowance. Unused minutes do not roll over. ## Automated Pro Unlimited web and app testing. Same parallelism as Automated, with no cap on automated testing minutes. | Parallel tests | Automated minutes / month | Annual plan (per month) | Month to month | | -------------- | ------------------------- | ----------------------- | -------------- | | 1 | Unlimited | $90 | $120 | | 2 | Unlimited | $180 | $240 | | 4 | Unlimited | $360 | $480 | | 8 | Unlimited | $720 | $960 | | 16 | Unlimited | $1440 | $1920 | | 24 | Unlimited | $2160 | $2880 | ## Enterprise Enterprise-grade testing for large teams. Starts at 10+ Parallel Tests, priced per quote. For Premium features & Enterprise-grade security: unlimited testing, Single Sign-On (SAML), priority support, and a private device cloud. See https://testingbot.com/enterprise or contact sales at https://testingbot.com/contact/new. ## Pay As You Go Credit bundles for teams that do not want a monthly commitment. Bought once, never expire, and can be topped up automatically when credits run low. | Bundle | Testing minutes | Price (one-time) | Parallel sessions | Cost per minute | | ------ | --------------- | ---------------- | ----------------- | --------------- | | Starter | 1,000 | $60 | 1 | $0.06 | | Growth | 2,000 | $120 | 2 | $0.06 | | Boost | 5,000 | $250 | 5 | $0.05 | | Professional | 15,000 | $600 | 10 | $0.04 | Pay As You Go covers automated and live manual testing on all desktop browsers, real iOS and Android devices, and iOS and Android simulators and emulators. Video recordings, local testing and 24/7 support are included. ## All plans include - Over 6100+ real desktop browsers and physical mobile devices - Pristine VMs with real browsers, and a range of display resolutions - Video recordings and screenshots of every test - Local testing (tunnel) for sites behind a firewall - Team management, and an open REST API for CI/CD - Android and iOS native app testing - Tech support and documentation ## Frequently asked questions ### Do I need a credit card to register? No, you do not need a credit card. Our 14-day free plan lets you try the service without paying anything. When you are ready to upgrade you can pay by credit card or wire transfer. ### Can I cancel my subscription at any time? Yes. Subscriptions can be cancelled at any time. If you cancel mid-month you will be charged for that month. ### What is a parallel test? Parallel tests are how many tests you can run at the same time. More parallel tests means a shorter total test run. Use our [Parallel Test Calculator](https://testingbot.com/support/parallel-calculator) to estimate the right number for your suite. ### Do you have a free open-source plan? Yes, open-source projects can apply for a free TestingBot plan. [Sign up for our open-source plan](https://testingbot.com/open-source). ### What happens if I exceed my monthly testing time? Once you reach your monthly testing time you can no longer run tests until your next billing month, or you can upgrade to a larger plan. ### What forms of payment do you accept? We accept credit cards and wire transfers. EU customers also have access to additional payment options. [Contact us](https://testingbot.com/contact/new) for wire transfer. ### Can I get a private cloud or on-prem deployment? Our Enterprise plan supports private cloud. We do not yet support on-premise deployments. [See enterprise features](https://testingbot.com/enterprise) or contact us for a custom quote. ### Do you offer high-volume packages? Yes, for very large test suites we offer custom high-volume packages with discounted per-minute rates. [Contact us](https://testingbot.com/contact/new) with your expected monthly volume. ### How is device time calculated? Device usage is counted from the time your app or browser is started on the device, until you stop the test. We do not charge for device startup, cleaning or preparation. For example: if you you run 1 test for 20 minutes, or 5 tests for 4 minutes, both times 20 minutes will be deducted from your plan. ### Do unused device minutes roll over to the next billing month? Unused minutes do not roll over to your next billing month. At the start of your next billing month, the minutes for your account are set to their default value. ### What happens when I upgrade or downgrade? If you decide to upgrade your account, the upgrade will take effect immediately. The unused portion of your current plan is automatically credited towards the new plan, so you only pay the difference. Downgrades take effect on the next billing cycle. ## Links - Pricing page: https://testingbot.com/pricing - Enterprise features: https://testingbot.com/enterprise - Free plan for open-source projects: https://testingbot.com/open-source - Parallel test calculator: https://testingbot.com/support/parallel-calculator - Contact sales: https://testingbot.com/contact/new --- URL: https://testingbot.com/newsletter # The TestingBot newsletter New browsers and devices as they land in the cloud, product changes worth knowing about, and the occasional testing tip. Written by the people who build it. - A few emails a year, never a drip campaign - One click to stop, from every email - Your address is never shared or sold ## Recently - Oct 1 Test on the iPhone Duo simulator before it ships - Aug 11 Vibium browser automation - Aug 5 Samsung Galaxy Tab S11 in the real device cloud [Let me read some first](https://testingbot.com/product-updates)  ·  [RSS](https://testingbot.com/product-updates.atom) ## Subscribe We will send one email to check the address is yours. Nothing else happens until you click the link in it. Email address Subscribe By subscribing you agree that we may email you about TestingBot. We record the date, source and IP of your confirmation, as we are required to. See our [privacy policy](https://testingbot.com/privacy). --- URL: https://testingbot.com/enterprise Enterprise plan # Test at enterprise scale, secured for the way you ship. TestingBot powers cross-browser, mobile and AI-driven testing for global engineering teams. Single-use firewalled VMs, SSO, auto-provisioning and a private device cloud, on European-hosted infrastructure with a 99.9% uptime SLA. [Contact Sales](https://testingbot.com/contact/new) [Book a demo](https://testingbot.com/demo) - **Browsers & devices:** 6100+ - **Uptime SLA:** 99.9% - **Years in business:** 14 Enterprise features 1. Priority SLA 2. Security 3. Data privacy 4. Private devices Priority support · 24/7 SLA Response \< 1 hour Uptime SLA 99.9% Channels Slack · Chat · Email SK Customer Success Manager INC-2917 · acknowledged in 2 min custom SLA · dedicated CSM · custom legal terms Your network 10.0.0.0/16 ![TestingBot logo](https://testingbot.com/assets/logo-head-bff10f1a578795c03fad4e7adbc3f173ce864715c8ebf3d147af75b5d05c00b0.png) TestingBot grid TLS 1.3 IP whitelisted Tunnel up Data residency · EU session-2917.mp4 12.4 MB console.log 238 KB screenshots/\*.png 4.1 MB encrypted at rest · per-test retention Private device cloud Reserved iPhone 17 iOS 26 Reserved iPad Pro iPadOS 18 Reserved Galaxy S26 Android 16 dedicated to your team · never shared Custom SLA · 24/7 priority support · dedicated CSM Encrypted tunnel · IP whitelisting · inbound + outbound European-hosted · GDPR · per-test retention windows Reserved exclusively for your team, never shared Trusted by engineering teams worldwide Built for security & scale ## Everything large engineering teams need from a testing cloud. Enterprise-grade controls layered on top of every TestingBot session: security, identity, governance and uninterrupted scale. - ### Single-Use VMs Every test runs on a fresh, isolated, firewalled VM that is wiped immediately after the session. Customize the OS or pre-installed software per run. - ### Network Security IP whitelisting (inbound and outbound) and the encrypted TestingBot Tunnel keep test traffic between your private network and our grid fully secured. - ### Data Control & Privacy Disable logging or recording per test. Pick exact data retention windows per asset type. All test artefacts stay on European-hosted infrastructure. - ### SSO & 2FA SAML 2.0 SSO with Okta, Azure AD, OneLogin and any custom SAML provider. Enforce 2FA across your entire organisation. - ### Auto-provisioning Onboard and offboard team members through your Identity Provider. Roles, teams and product access stay in sync automatically. - ### Sub-teams & Quotas Control access with role-based permissions (separate IAM and RBAC roles plus custom roles). Compliance & assurance ## Built and operated to standards procurement teams already trust. EU-only data residency, a published CSA STAR self-assessment and a public Trust Center keep your security review on the fast track. [Trust Center](https://trust.testingbot.com) - ![CSA STAR Level 1](https://testingbot.com/assets/security/star-11edcc06553505b9273f1b262013331690e0898a62e098ea785ef5781f32a53e.png) CSA STAR Level 1 Cloud Security Alliance - ![GDPR](https://testingbot.com/assets/security/gdpr-7823fb95f16df44e3d7aa0c6b0541d66fce3db81ce70907bc8fad94d0b0ac1e6.png) GDPR EU data protection - ![CCPA](https://testingbot.com/assets/security/ccpa-1d50dd27aea6947f5717792a2b4b32b38ff5f8a8d899884038fa68fe4b8ee0f5.svg) CCPA California privacy - ISO ISO 27001 Aligned controls - CSA CSA STAR Level 1 Published self-assessment (CAIQ) - European-hosted Dedicated devices ## Private Device Cloud With Private Device Cloud, we take care of the setup, configuration and maintenance of dedicated devices exclusively for your company. - Unlike our Public Device Cloud, these devices are reserved only for your team, ensuring full control and privacy. - Select your preferred devices, and we will make them available in our data center for seamless testing. - Run both automated and manual tests on your private iOS and Android devices with full control and security. [Learn more](https://testingbot.com/enterprise/private-device-cloud) ![Private Device Cloud](https://testingbot.com/assets/enterprise/private-01e26e5dad16fa4b9b824124cec4112843b10fdf88c7432e8c0c0e873bab5900.webp) Side by side ## How TestingBot for Enterprise compares. Everything in Standard, plus the security, governance and support large teams need. | Feature | Standard Plans | Enterprise Plans Recommended | | --- | --- | --- | | Security & Compliance | | IP Whitelisting (inbound & outbound) | – | | | Single-Use VMs + Firewalled | | | | 2FA enforcement | – | | | Administration | | SAML 2.0 Single sign-on (SSO) | – | | | Role-based access control & custom roles | – | | | Analytics per team & team-member | – | | | Support | | Dedicated Support person & Customer Success Manager | – | | | Custom SLA | – | | | 24/7 priority support via phone and e-mail | – | | | Custom Legal Terms | – | | | Product | | Customize OS / Pre-installed Software | – | | | Early access to beta features | – | | | Unlimited Users | – | | ### Ready to roll out TestingBot across your organisation? Talk to our enterprise team about SSO, sub-teams, dedicated devices and a deployment plan that fits your security review. [Contact Sales](https://testingbot.com/contact/new) [Book a demo](https://testingbot.com/demo) --- URL: https://testingbot.com/enterprise/private-device-cloud Dedicated devices # Private Real Device Cloud, dedicated to your team. Pick the exact iOS and Android devices you want and we provision them in our datacenter. Reserved 24/7 for your team, customised to your testing needs and reachable on real carrier networks. [Contact Sales](https://testingbot.com/contact/new) [Book a demo](https://testingbot.com/demo) - **Devices on offer:** Any iOS / Android - **Availability:** 24/7 ![Private Real Device Cloud, dedicated to your team.](https://testingbot.com/assets/enterprise/private/hero-42a08a4588b3ee004200a9f7c0592ae09aceb416aa5eee61b878a404ea4af3a2.png) Trusted by engineering teams worldwide What dedicated devices unlock ## The exact device matrix your customers use: at your fingertips, around the clock. We buy, configure and host the iOS and Android devices you choose. Reserved exclusively for your team and customised to fit how you actually test. - ### Pick the exact devices Choose the iPhone, iPad, Pixel, Galaxy or any other device that matches your customer base. We source it, image it and install it for you. - ### Reserved for your team Your devices are never shared with other customers. No waiting in queues, guaranteed availability. - ### Real carrier networks Equip devices with a physical SIM or eSIM to test scenarios Wi-Fi cannot reach: SMS flows, phone calls, mobile-data behaviour and roaming. - ### Custom device profiles We pre-install your apps, profiles, certificates and beta builds. Keep persistent state across runs when you need it. - ### Available 24 / 7 Your dedicated devices are reachable around the clock from any region for both automated runs and manual live sessions. - ### Secure & European-hosted Devices live in our locked, European-hosted datacenter. Network access stays inside your tunnel; logs and recordings stay on EU infrastructure. ![Real devices, in our secure EU datacenter.](https://testingbot.com/assets/enterprise/datacenter-d4fc366e9a4e8a106ec41e6817e21adb2aeb366a397ac258fd29ba9cd4de72c4.jpg) How we host your devices ## Real devices, in our secure EU datacenter. We purchase the iOS and Android devices you specify, install and configure them in our European-hosted datacenter, and connect them to our cloud so your team can run automated and manual tests around the clock. - Your model list, sourced and provisioned by us, including legacy or hard-to-find devices. - Custom OS settings, certificates, profiles, beta apps and persistent state, set up once and re-applied automatically. - Devices stay reachable 24/7. Carrier connectivity ## Test on real carrier networks with SIM and eSIM. Equip your dedicated devices with physical SIMs or eSIMs and exercise the full set of mobile-network behaviours your customers actually depend on. - Send and receive real SMS, including 2FA codes, magic links and OTP flows. - Place and receive real phone calls; verify in-app calling, voicemail and call-state UI. - Validate mobile-data, low-signal and roaming behaviours that Wi-Fi can't reproduce. ![Test on real carrier networks with SIM and eSIM.](https://testingbot.com/assets/enterprise/sims-ef855544bead23d65f39cdc10fba272c2a93d82fa0e28026ab6ea10fd91131e6.jpg) Frequently asked ## Private Device Cloud: questions teams ask us. Everything procurement, security and engineering tend to want to know before rolling out dedicated devices. What is a Private Device Cloud? A Private Device Cloud is a set of real iOS and Android devices that TestingBot purchases, hosts and maintains in our EU datacenter exclusively for your team. The devices are reachable 24/7 over the same Selenium and Appium endpoints you already use, but no other customer can ever run a test on them. Can we choose the exact device models we want? Yes. You pick the exact iPhone, iPad, Pixel, Galaxy or other models you need, including specific OS versions, regional variants and legacy devices that have left our public cloud. If we don't already stock the model, we will source it for you. Do dedicated devices support real carrier networks? Yes. We can equip your devices with physical SIMs or eSIMs from your preferred carriers, so you can test SMS, phone calls, mobile data, low-signal and roaming behaviour just like an end user would experience them. Can we customise OS, profiles or pre-installed apps? We pre-install your apps and beta builds, deploy custom configuration profiles, certificates and entitlements, and adjust device-level settings to match your needs. By default we wipe state between tests, but on private devices we can keep persistent state when that's useful. Where are the devices hosted and how are they secured? All Private Device Cloud devices are hosted in our EU-based datacenter. Devices are physically locked, network access is restricted, and logs, screenshots and video recordings stay on EU infrastructure. We are GDPR compliant and CSA STAR Level 1 registered. How is the Private Device Cloud priced? Pricing is custom and depends on the number of devices, models, carrier connectivity and the support tier you need. Talk to our enterprise team, we will put together a quote that fits your scope and timeline. ### Tell us which devices you need. Share the device matrix, the testing scenarios and the timeline. Our enterprise team will draft a Private Device Cloud setup that fits. [Contact Sales](https://testingbot.com/contact/new) [Book a demo](https://testingbot.com/demo) --- URL: https://testingbot.com/partners # Partner with TestingBot Join our partner ecosystem to expand your business opportunities and enhance your product capabilities with TestingBot's next generation testing platform. [Technology Partners](https://testingbot.com/partners/technology-partners) [Reseller Partners](https://testingbot.com/partners/reseller-partners) - 1+bn Tests Executed - 180k Active Users - 5,500+ Browsers & Devices ![Partner with TestingBot](https://testingbot.com/assets/hero/automated-f13c4f47b8139a589966d59465e1aae2e65dae155b106d3956ce990a302a80a4.webp) ![Partner](https://testingbot.com/assets/integrations/zebrunner-51fe5d07743785e67d69f0c97a8ae327e89a01545a54662e5cf1cf3c24601bf4.svg) ![Partner](https://testingbot.com/assets/integrations/qmetry-0647488fc2248e9e180e384e614b8cdbd1e8e4cf5360ab450d65a512490b0405.svg) ![Partner](https://testingbot.com/assets/integrations/semrush-6ea828d5e278d18118745e756c0e8359432ccbc3834045c57b6818f39d21ab92.webp) ![Partner](https://testingbot.com/assets/integrations/aws-37fe010e66887d20cd831130e90da33a0ec731b5dd87fbba8adc933db17b0709.webp) ![Partner](https://testingbot.com/assets/integrations/jira-63eb4095ef566b1fe712e84f18e56b37ea706defb2b21450d93f1304c9886fa7.svg) ![Partner](https://testingbot.com/assets/integrations/slack-935c2233d65c4fd7ce6ac282c0abe685fcc2ed73d5d814083a071ecd4557befa.webp) ![Partner](https://testingbot.com/assets/integrations/bugsnag-3e0b2bebfb3c599b8d986dda1bd59712ae70de4a9450b7522a698ad786ff6e62.svg) ![Partner](https://testingbot.com/assets/integrations/bamboo-c8fe09a3cb0b74d6055e3aa6607d77720bfe29ff487b8e3cf825138adf634346.svg) ![Partner](https://testingbot.com/assets/integrations/circleci-ede029654ff39b60342e4a5bba8fc9a3a49305cc38bda215b7cf8d07e85e0543.webp) ![Partner](https://testingbot.com/assets/integrations/zebrunner-51fe5d07743785e67d69f0c97a8ae327e89a01545a54662e5cf1cf3c24601bf4.svg) ![Partner](https://testingbot.com/assets/integrations/qmetry-0647488fc2248e9e180e384e614b8cdbd1e8e4cf5360ab450d65a512490b0405.svg) ![Partner](https://testingbot.com/assets/integrations/semrush-6ea828d5e278d18118745e756c0e8359432ccbc3834045c57b6818f39d21ab92.webp) ![Partner](https://testingbot.com/assets/integrations/aws-37fe010e66887d20cd831130e90da33a0ec731b5dd87fbba8adc933db17b0709.webp) ![Partner](https://testingbot.com/assets/integrations/jira-63eb4095ef566b1fe712e84f18e56b37ea706defb2b21450d93f1304c9886fa7.svg) ![Partner](https://testingbot.com/assets/integrations/slack-935c2233d65c4fd7ce6ac282c0abe685fcc2ed73d5d814083a071ecd4557befa.webp) ![Partner](https://testingbot.com/assets/integrations/bugsnag-3e0b2bebfb3c599b8d986dda1bd59712ae70de4a9450b7522a698ad786ff6e62.svg) ![Partner](https://testingbot.com/assets/integrations/bamboo-c8fe09a3cb0b74d6055e3aa6607d77720bfe29ff487b8e3cf825138adf634346.svg) ![Partner](https://testingbot.com/assets/integrations/circleci-ede029654ff39b60342e4a5bba8fc9a3a49305cc38bda215b7cf8d07e85e0543.webp) ## Partner Programs Choose the partnership that best fits your business needs and goals. ### Technology Partners TestingBot platform integrates with tools that developers use daily. Partner with us to enhance your product capabilities, reduce interruptions for developers and expand customer reach. - API integration support - Co-marketing opportunities - Technical support and documentation [Learn More](https://testingbot.com/partners/technology-partners) ### Reseller Partners Expand your business by partnering with TestingBot. Join our Partner Program and earn lucrative commissions, receive dedicated support and gain recognition for your sales success. - Competitive commission rates - Dedicated partner support - Sales and marketing materials [Learn More](https://testingbot.com/partners/reseller-partners) ### Ready to Partner with Us? Join TestingBot's growing partner ecosystem and unlock new opportunities for your business. [See existing integrations](https://testingbot.com/integrations) --- URL: https://testingbot.com/partners/technology-partners # Technology Partners TestingBot platform integrates with tools that developers use daily. Partner with us to enhance your product capabilities, reduce interruptions for developers and expand customer reach. [Apply now](https://testingbot.com#apply) [All Partner Programs](https://testingbot.com/partners) ![Technology Partners](https://testingbot.com/assets/hero/automated-f13c4f47b8139a589966d59465e1aae2e65dae155b106d3956ce990a302a80a4.webp) ![Partner](https://testingbot.com/assets/integrations/zebrunner-51fe5d07743785e67d69f0c97a8ae327e89a01545a54662e5cf1cf3c24601bf4.svg) ![Partner](https://testingbot.com/assets/integrations/qmetry-0647488fc2248e9e180e384e614b8cdbd1e8e4cf5360ab450d65a512490b0405.svg) ![Partner](https://testingbot.com/assets/integrations/semrush-6ea828d5e278d18118745e756c0e8359432ccbc3834045c57b6818f39d21ab92.webp) ![Partner](https://testingbot.com/assets/integrations/aws-37fe010e66887d20cd831130e90da33a0ec731b5dd87fbba8adc933db17b0709.webp) ![Partner](https://testingbot.com/assets/integrations/jira-63eb4095ef566b1fe712e84f18e56b37ea706defb2b21450d93f1304c9886fa7.svg) ![Partner](https://testingbot.com/assets/integrations/slack-935c2233d65c4fd7ce6ac282c0abe685fcc2ed73d5d814083a071ecd4557befa.webp) ![Partner](https://testingbot.com/assets/integrations/bugsnag-3e0b2bebfb3c599b8d986dda1bd59712ae70de4a9450b7522a698ad786ff6e62.svg) ![Partner](https://testingbot.com/assets/integrations/bamboo-c8fe09a3cb0b74d6055e3aa6607d77720bfe29ff487b8e3cf825138adf634346.svg) ![Partner](https://testingbot.com/assets/integrations/circleci-ede029654ff39b60342e4a5bba8fc9a3a49305cc38bda215b7cf8d07e85e0543.webp) ![Partner](https://testingbot.com/assets/integrations/zebrunner-51fe5d07743785e67d69f0c97a8ae327e89a01545a54662e5cf1cf3c24601bf4.svg) ![Partner](https://testingbot.com/assets/integrations/qmetry-0647488fc2248e9e180e384e614b8cdbd1e8e4cf5360ab450d65a512490b0405.svg) ![Partner](https://testingbot.com/assets/integrations/semrush-6ea828d5e278d18118745e756c0e8359432ccbc3834045c57b6818f39d21ab92.webp) ![Partner](https://testingbot.com/assets/integrations/aws-37fe010e66887d20cd831130e90da33a0ec731b5dd87fbba8adc933db17b0709.webp) ![Partner](https://testingbot.com/assets/integrations/jira-63eb4095ef566b1fe712e84f18e56b37ea706defb2b21450d93f1304c9886fa7.svg) ![Partner](https://testingbot.com/assets/integrations/slack-935c2233d65c4fd7ce6ac282c0abe685fcc2ed73d5d814083a071ecd4557befa.webp) ![Partner](https://testingbot.com/assets/integrations/bugsnag-3e0b2bebfb3c599b8d986dda1bd59712ae70de4a9450b7522a698ad786ff6e62.svg) ![Partner](https://testingbot.com/assets/integrations/bamboo-c8fe09a3cb0b74d6055e3aa6607d77720bfe29ff487b8e3cf825138adf634346.svg) ![Partner](https://testingbot.com/assets/integrations/circleci-ede029654ff39b60342e4a5bba8fc9a3a49305cc38bda215b7cf8d07e85e0543.webp) ## Partner Benefits Join our technology partner program and unlock powerful integrations with TestingBot's comprehensive testing platform. ### API Integration Support Get comprehensive technical documentation, SDKs and dedicated engineering support for seamless integration. ### Co-Marketing Opportunities Joint marketing initiatives, case studies, webinars and placement in our partner directory to expand your reach. ### Technical Documentation Access to comprehensive technical resources, integration guides and best practices documentation. ### Partner Portal Access Dedicated partner portal with resources, training materials and integration tracking tools. ### Priority Support Dedicated technical support channel with priority response times for integration issues. ### Partner Recognition Featured placement in our partner ecosystem, case studies and recognition at industry events. ## Integration Opportunities Partner with TestingBot to create powerful integrations that enhance developer workflows and provide seamless testing experiences. - **Integrate into your product:** Embed our testing capabilities into your app. Offer automated testing to your customers through TestingBot's whitelabel automated testing offerings, or embed realtime live testing in your product. - **Developer Advocate:** Help developers use TestingBot's platform. We supply you with documentation, tools and free extended trial credits. - **Integrate your product with TestingBot:** Create an integration between TestingBot and your product. ![Technology Integration](https://testingbot.com/assets/features/coverage-d16d677b6a3ac8fac706949abfa45577d35762f298030c3f59a561e7b5fbd8b4.jpg) ## Apply to Become a Technology Partner Ready to integrate with TestingBot? Fill out the form below and our partnership team will be in touch. Company Name \* Email Address \* Tell us about your integration \* Submit Application --- URL: https://testingbot.com/partners/reseller-partners # Reseller Partners Expand your business by partnering with TestingBot. Resellers in the Partner Program enjoy lucrative commissions, dedicated support and recognition for their sales excellence. [Apply now](https://testingbot.com#apply) [All Partner Programs](https://testingbot.com/partners) ![Reseller Partners](https://testingbot.com/assets/hero/automated-f13c4f47b8139a589966d59465e1aae2e65dae155b106d3956ce990a302a80a4.webp) ![Partner](https://testingbot.com/assets/integrations/zebrunner-51fe5d07743785e67d69f0c97a8ae327e89a01545a54662e5cf1cf3c24601bf4.svg) ![Partner](https://testingbot.com/assets/integrations/qmetry-0647488fc2248e9e180e384e614b8cdbd1e8e4cf5360ab450d65a512490b0405.svg) ![Partner](https://testingbot.com/assets/integrations/semrush-6ea828d5e278d18118745e756c0e8359432ccbc3834045c57b6818f39d21ab92.webp) ![Partner](https://testingbot.com/assets/integrations/aws-37fe010e66887d20cd831130e90da33a0ec731b5dd87fbba8adc933db17b0709.webp) ![Partner](https://testingbot.com/assets/integrations/jira-63eb4095ef566b1fe712e84f18e56b37ea706defb2b21450d93f1304c9886fa7.svg) ![Partner](https://testingbot.com/assets/integrations/slack-935c2233d65c4fd7ce6ac282c0abe685fcc2ed73d5d814083a071ecd4557befa.webp) ![Partner](https://testingbot.com/assets/integrations/bugsnag-3e0b2bebfb3c599b8d986dda1bd59712ae70de4a9450b7522a698ad786ff6e62.svg) ![Partner](https://testingbot.com/assets/integrations/bamboo-c8fe09a3cb0b74d6055e3aa6607d77720bfe29ff487b8e3cf825138adf634346.svg) ![Partner](https://testingbot.com/assets/integrations/circleci-ede029654ff39b60342e4a5bba8fc9a3a49305cc38bda215b7cf8d07e85e0543.webp) ![Partner](https://testingbot.com/assets/integrations/zebrunner-51fe5d07743785e67d69f0c97a8ae327e89a01545a54662e5cf1cf3c24601bf4.svg) ![Partner](https://testingbot.com/assets/integrations/qmetry-0647488fc2248e9e180e384e614b8cdbd1e8e4cf5360ab450d65a512490b0405.svg) ![Partner](https://testingbot.com/assets/integrations/semrush-6ea828d5e278d18118745e756c0e8359432ccbc3834045c57b6818f39d21ab92.webp) ![Partner](https://testingbot.com/assets/integrations/aws-37fe010e66887d20cd831130e90da33a0ec731b5dd87fbba8adc933db17b0709.webp) ![Partner](https://testingbot.com/assets/integrations/jira-63eb4095ef566b1fe712e84f18e56b37ea706defb2b21450d93f1304c9886fa7.svg) ![Partner](https://testingbot.com/assets/integrations/slack-935c2233d65c4fd7ce6ac282c0abe685fcc2ed73d5d814083a071ecd4557befa.webp) ![Partner](https://testingbot.com/assets/integrations/bugsnag-3e0b2bebfb3c599b8d986dda1bd59712ae70de4a9450b7522a698ad786ff6e62.svg) ![Partner](https://testingbot.com/assets/integrations/bamboo-c8fe09a3cb0b74d6055e3aa6607d77720bfe29ff487b8e3cf825138adf634346.svg) ![Partner](https://testingbot.com/assets/integrations/circleci-ede029654ff39b60342e4a5bba8fc9a3a49305cc38bda215b7cf8d07e85e0543.webp) ## Reseller Benefits Join our reseller program and unlock revenue opportunities with comprehensive support and competitive commission structures. ### Competitive Commissions Earn attractive commission rates on all TestingBot subscriptions you refer, with performance-based bonuses for top performers. ### Dedicated Support Get priority access to our partner success team for sales support, technical questions, and customer assistance. ### Sales & Marketing Materials Access to professional sales collateral, presentations, case studies, and co-branded marketing materials. ### Training & Certification Comprehensive product training, sales certification programs, and ongoing education to maximize your success. ### Partner Portal Access to a comprehensive partner portal with commission tracking, customer management, and resource library. ### Partner Recognition Top performers receive recognition awards, conference invitations, and exclusive networking opportunities. ## Who We're Looking For We partner with organizations that share our commitment to quality and customer success. There is no partnership fee. ### Ideal Partner Profile - Software consultancies and development agencies - QA and testing service providers - DevOps and infrastructure consultants - Technology solution providers ## Apply to Become a Reseller Partner Ready to join our reseller program? Fill out the form below and our partnerships team will review your application. Company Name \* Email Address \* Tell us about your business \* Submit Application --- URL: https://testingbot.com/open-source Free for Open-Source # TestingBot loves open source You move new ideas forward and make software better for everyone. Get free, unlimited cross browser and mobile testing for your open source project. [Apply for free access](https://testingbot.com/users/sign_up) [See what's included](https://testingbot.com#whats-included) 1+bn Tests Executed 180k Active Users 6100+ Browsers & Devices Trusted by some of the world's most innovative companies What you get ## What the TestingBot Open Source plan includes TestingBot's Open Source program offers free, unlimited cross browser and mobile testing to qualifying open source projects, including 2 concurrent tests on 6100+ real browsers and real iOS and Android devices, with support for Selenium, Cypress, Playwright and Appium. ### Unlimited testing Run as many automated tests as you need. No monthly cap, ever. ### 2 parallel sessions Two concurrent tests so your CI pipeline keeps moving. ### 6100+ browsers & devices Every major browser plus physical iOS and Android devices. ### AI, codeless, video & CI Codeless and AI test creation, video, screenshots, logs and badges, included. - Free forever for active open source projects - Unlimited automated test runs - 2 concurrent (parallel) tests - 6100+ browser and OS combinations - Real iOS and Android devices - Selenium, Cypress, Playwright, Puppeteer, Appium, Espresso, XCUITest - Video, screenshots, console and network logs - CI/CD support, status badges and a browser matrix ## Which frameworks and devices are supported Keep your existing test suite. Point it at the TestingBot grid and run it for free. ### Web frameworks Run cross browser tests with the tools you already use. - [![Selenium logo](https://testingbot.com/assets/integrations/selenium-1403d7bf83f64f56f05cea3fe50ae45bc3025d5a2433b7b05d2b199fec753cf6.svg)Selenium](https://testingbot.com/support/web-automate/selenium) - [![Cypress logo](https://testingbot.com/assets/support/cypress-409c9fd3f79115ed7676aa1df4f6a217dc4402b3366a61984924aec7b4fefa97.svg)Cypress](https://testingbot.com/support/web-automate/cypress) - [![Playwright logo](https://testingbot.com/assets/support/playwright-2dcb4e75b55ac0d1135cfa47be6137b13f46a2047ce78b96a34f51b0e2cc0d96.svg)Playwright](https://testingbot.com/support/web-automate/playwright) - [![Puppeteer logo](https://testingbot.com/assets/integrations/puppeteer-7bb02294d7d9ef0b92e56f08d1bf99574298111ee79de39d298ea99efb17f322.svg)Puppeteer](https://testingbot.com/support/web-automate) ### Mobile and app testing Test on [real iOS and Android devices](https://testingbot.com/support/app-automate). - [![Appium logo](https://testingbot.com/assets/integrations/appium-75e88eb18ca3b2ce63d641547ce06398c8ecd971ed17187c9134c8d205465f18.svg)Appium](https://testingbot.com/support/app-automate) - [![Espresso logo](https://testingbot.com/assets/integrations/espresso-a8ff30da0c385f89f4e694bb29fc4779ad132e8e3f4a777a38e608e0492ff610.svg)Espresso](https://testingbot.com/support/app-automate) - [![XCUITest logo](https://testingbot.com/assets/integrations/xcuitest-4a935f2368661d89d3559b3c5af3377236029478ae76bb43253f3c3bee712096.svg)XCUITest](https://testingbot.com/support/app-automate) - [![Maestro logo](https://testingbot.com/assets/integrations/maestro-9d782f30f13fb4efaf9928b704c7a787eb14cb9e5b20c650137bb65892334b8b.svg)Maestro](https://testingbot.com/support/app-automate) ### Codeless and AI testing No script to write? Create and run tests in plain language. - [AI test creation](https://testingbot.com/support/ai) - [Codeless testing](https://testingbot.com/features/ai-testing) ## Who qualifies for free open source testing The Open Source plan is for public, non-commercial projects that the community can use and contribute to. Not sure if your project qualifies? Apply anyway, we review every request. - Publicly hosted repository on GitHub, GitLab or Bitbucket - An OSI-approved license (MIT, Apache 2.0, GPL, BSD and similar) - Actively maintained, with recent commits - A clear README describing the project - Not a commercial or closed-source product ## How to apply for the Open Source plan Three steps and you are testing for free, usually within a couple of business days. 1 ### Create a free account Sign up for the free plan, no credit card required. 2 ### Submit your project Tell us your repository URL and a short description. 3 ### Get approved We verify and upgrade your account, usually within a couple of business days. First, create your free account. Once you are signed in, you can submit your project right here. [Create free account](https://testingbot.com/users/sign_up) Already have an account? [Sign in](https://testingbot.com/users/sign_in). ## Everything you need to test with confidence ### Cross browser grid Make sure your project works everywhere, across 6100+ browser and OS combinations. [View supported browsers](https://testingbot.com/support/web-automate/browsers) ### CI/CD integration Plug TestingBot into your pipeline with GitHub Actions, Jenkins, [Travis CI](https://testingbot.com/support/integrations/ci-cd/travis-ci) or [CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci). [See integrations](https://testingbot.com/support/integrations) ### Debugging artifacts Every test ships with a video, screenshots, and console and network logs so you can debug failures fast. [Explore features](https://testingbot.com/features) ### Status badges & browser matrix Show your test results on your README with [status badges](https://testingbot.com/support/other/status-badges) and a [browser matrix](https://testingbot.com/support/other/browser-matrix). [Add a badge](https://testingbot.com/support/other/status-badges) ## How TestingBot compares for open source - **Price for open source:** Free - **Concurrent tests:** 2 - **Browser and device combinations:** 6100+ - **Real mobile devices:** Yes, iOS and Android - **Frameworks:** Selenium, Cypress, Playwright, Puppeteer, Appium, Espresso, XCUITest - **Hosting and compliance:** European-hosted, GDPR-compliant - **Free alternative to:** BrowserStack and Sauce Labs Want the full breakdown? [Compare plans and pricing](https://testingbot.com/pricing). ## Frequently asked questions ### Is TestingBot really free for open source projects? Yes. Qualifying open source projects get a free TestingBot Open Source plan with unlimited automated testing and 2 concurrent tests, at no cost. The plan stays free as long as your project remains an active, public open source project. ### Which testing frameworks does the open source plan support? The plan supports Selenium, Cypress, Playwright and Puppeteer for web, and Appium, Espresso, XCUITest and Maestro for mobile apps, plus codeless and AI test creation. You use the same TestingBot cloud grid as paid plans. ### Which browsers and devices can I test on for free? Open source projects can run tests on 6100+ browser and operating system combinations and on real iOS and Android devices in the TestingBot cloud, including older and current browser versions. ### How do I qualify for the open source program? Your project must be publicly hosted (for example on GitHub, GitLab or Bitbucket), use an OSI-approved open source license, be non-commercial and be actively maintained. Sign up for a free trial, then submit your repository URL and a short description on this page. ### How many concurrent tests do I get on the open source plan? The open source plan includes 2 concurrent tests, which lets you run tests in parallel and integrate with CI systems such as GitHub Actions, Travis CI and CircleCI. ### Can I use TestingBot with GitHub Actions and other CI systems? Yes. The open source plan works with any CI/CD system. Point your existing Selenium, Cypress, Playwright or Appium tests at the TestingBot grid, and add status badges and a browser matrix to your README. ### Is TestingBot a free alternative to BrowserStack and Sauce Labs for open source? Yes. TestingBot offers open source maintainers a free alternative to BrowserStack and Sauce Labs for cross browser and real device testing, with EU-hosted, GDPR-compliant infrastructure. ### What do I get with each test run? Every test includes a video recording, screenshots, network and console logs and metadata so you can debug failures, plus shareable public links and embeddable status badges for your project page. ## Ready to test your open source project for free? Apply today and ship with confidence on every browser and device. [Apply for free access](https://testingbot.com/users/sign_up) --- URL: https://testingbot.com/about # About Us Founded in 2011, TestingBot aims to provide an easy-to-use platform for cross-browser and mobile testing. By leveraging the power of cloud computing, we can scale effortlessly to meet your testing needs. We offer flexible plans tailored for both startups and large enterprises, enabling you to test your website across hundreds of real browser and mobile device combinations. Together with **excellent customer service** , we're confident you will enjoy testing your websites with TestingBot. ![Helping developers and teams test. Reliably and at scale.](https://testingbot.com/assets/about/bg-2e0825c48447a2dbdefc8bb7e4bfe93c3fa7d218def5e88141411eb056d2c813.jpg) ## Trusted by some of the world's most innovative companies ## Helping developers and teams test. Reliably and at scale. TestingBot provides a powerful online testing grid with a wide range of real browsers and physical mobile devices. Our entire infrastructure is hosted in multiple European data centers, located in Belgium, Germany, Finland and Switzerland, ensuring optimal security, performance and reliability. At TestingBot, we take pride in delivering a stable, fast and secure testing environment for our customers. +6 million Tests each month 99.9% Uptime 12,000 New users annually ## What drives us The principles that shape how we build, host and support the TestingBot platform. ### Security Security is a first-class feature at TestingBot. We guarantee single-use machines and wiped devices for each test. [Visit our Trust Center](https://trust.testingbot.com) ### European infrastructure Our platform runs from our own infrastructure across Europe. No third-party cloud providers, no data leaving the EU. Your tests and test data stay safe. ### Customer-first support Real engineers answer every support ticket, fast. We've been building TestingBot since 2011 and we still treat every customer like our first. ## Our journey TestingBot was founded in 2011 to solve a personal need: the founder wanted an easy way to run automated tests using Selenium on a website. Automated web testing was a tedious task, and the only solution was to set up a Selenium Grid on your own server. We wanted to change that and started TestingBot to provide a simple solution for automated web testing. Over the years, we've continuously expanded our platform with new features. Explore the timeline below for a complete overview of our journey. 1. 2011 🚀 TestingBot is released to the public as one of the first cloud-based Selenium grids. 2. 2012 Realtime manual testing was added for Desktop browsers. 3. 2013 🖥️ macOS based testing was released: the first company providing test access to remote Safari browsers. 4. 2014 iOS Simulators & Android Emulators are added for mobile testing. 5. 2015 TestingBot Tunnel is released: test websites running on your own computer or private network. 6. 2016 📱 We make our real device farm available to the public. Test your website on physical iOS and Android devices. 7. 2017 🗺️ This year we added geolocation testing, headless testing and various integrations with Slack, TeamCity etc. 8. 2018 We launch codeless automation. Import recorded tests from Selenium IDE and run the tests at specific intervals. 9. 2019 Live App Testing is launched: test your mobile app manually on (physical) mobile devices. 10. 2020 📈 Introducing Test Analytics: keep track of your test analytics. Release Cypress automation support, run your Cypress tests on TestingBot. 11. 2021 We've added Puppeteer Playwright testing to our features. Connect your Puppeteer scripts to the remote TestingBot browser grid. 12. 2022 Android Espresso and XCUITest support. Run tests against native apps on TestingBot's device grid. 13. 2023 Connect your Playwright scripts to the remote TestingBot browser grid. 14. 2024 🔎 Visual Regression Testing is launched: compare snapshots of your website or mobile app against baselines. 15. 2025 ✨ Introducing AI-based testing: describe in natural language what you would like to test. TestingBot will monitor your website for failures. 16. 2026 🎭 Maestro mobile testing support: run Maestro flows against iOS simulators, Android emulators and physical devices on TestingBot's device cloud. ## Our team ![Jochen Delabie](https://testingbot.com/assets/about/avatarj_300-2d6040c8da7195c628409e9c307f78c9a3350ed4378a7506b14b4cfab353b291.webp) ### [Jochen](https://testingbot.com/about/jochen-d) Founder ![Chandrika Deb](https://testingbot.com/assets/about/chandrika_300-a697ab8b715138aa1f5ad1e5090a3cf42cd4592340cd2f35c856ec3e70460b3b.webp) ### [Chandrika](https://testingbot.com/about/chandrika-deb) Technical Writer ![Laura](https://testingbot.com/assets/about/avatarl_300-9016c575ff384e5d4e40b185204942c8d6828ec793649b1c3f2309da1c7a0bc6.webp) #### Laura Support ![Sander](https://testingbot.com/assets/about/avatars_300-6ba3ad8b8a51453c3ac05fe1855e4c7ccee9c139a1333329d21775721db26f72.webp) #### Sander Marketing ## Want to join our team? Interested in working on exciting technologies? [We're hiring](https://testingbot.com/careers) --- URL: https://testingbot.com/about/jochen-d ![Jochen Delabie](https://testingbot.com/assets/about/avatarj_300-2d6040c8da7195c628409e9c307f78c9a3350ed4378a7506b14b4cfab353b291.webp) # Jochen Delabie Founder & CEO Moerbeke-Waas, Belgium Founder of TestingBot. Building cloud testing infrastructure since 2011. Jochen Delabie founded TestingBot in 2011 to make cross-browser and mobile testing approachable for development teams. Over the past 15+ years he has worked on test automation, infrastructure and developer tooling around Selenium, Appium and Playwright. He writes regularly on the TestingBot blog about browser automation, mobile testing and AI-driven testing workflows. ## Areas of expertise Cross browser testing Selenium WebDriver Appium Playwright AI-driven testing Mobile app testing Test automation Continuous integration ## Recent blog posts by Jochen [![Announcing AI Test Insights](https://testingbot.com/assets/blog/2026/ai-insights.jpg) ](https://testingbot.com/blog/ai-test-insights) Jun 4, 2026 ### [Announcing AI Test Insights](https://testingbot.com/blog/ai-test-insights) Use AI Insights to quickly discover why an automated test failed. [Read more: Announcing AI Test Insights](https://testingbot.com/blog/ai-test-insights) [![Maestro Physical Device Testing](https://testingbot.com/assets/blog/2026/maestro-physical.jpg) ](https://testingbot.com/blog/maestro-physical-device-testing) Jan 19, 2026 ### [Maestro Physical Device Testing](https://testingbot.com/blog/maestro-physical-device-testing) Maestro testing in parallel on physical Android and iOS devices. [Read more: Maestro Physical Device Testing](https://testingbot.com/blog/maestro-physical-device-testing) [![tvOS Physical Device Testing](https://testingbot.com/assets/blog/2025/appletv.jpg) ](https://testingbot.com/blog/tvos-automated-testing) Oct 15, 2025 ### [tvOS Physical Device Testing](https://testingbot.com/blog/tvos-automated-testing) Run automated tvOS tests on physical AppleTV devices. [Read more: tvOS Physical Device Testing](https://testingbot.com/blog/tvos-automated-testing) [![MacOS Tahoe Cloud Testing](https://testingbot.com/assets/blog/2025/tahoe.jpg) ](https://testingbot.com/blog/macos-tahoe-beta-testing) Jun 10, 2025 ### [MacOS Tahoe Cloud Testing](https://testingbot.com/blog/macos-tahoe-beta-testing) Test your websites and apps on Apple's upcoming macOS Tahoe (macOS 26). [Read more: MacOS Tahoe Cloud Testing](https://testingbot.com/blog/macos-tahoe-beta-testing) [![ChromeOS Cloud Testing](https://testingbot.com/assets/blog/2025/chromeos.jpg) ](https://testingbot.com/blog/chromeos-testing) Jun 10, 2025 ### [ChromeOS Cloud Testing](https://testingbot.com/blog/chromeos-testing) ChromeOS Manual and Automated testing in the cloud. [Read more: ChromeOS Cloud Testing](https://testingbot.com/blog/chromeos-testing) [![Accessibility testing with TestingBot](https://testingbot.com/assets/blog/2025/accessibility_testing.jpg) ](https://testingbot.com/blog/accessibility-web-testing) May 26, 2025 ### [Accessibility testing with TestingBot](https://testingbot.com/blog/accessibility-web-testing) TestingBot offers automated accessibility testing for your websites. [Read more: Accessibility testing with TestingBot](https://testingbot.com/blog/accessibility-web-testing) [![Monitor your Shopify Store](https://testingbot.com/assets/blog/2025/shopify-app.webp) ](https://testingbot.com/blog/shopify-store-monitoring) Feb 27, 2025 ### [Monitor your Shopify Store](https://testingbot.com/blog/shopify-store-monitoring) Protect your Shopify Store from downtime: automated monitoring of Shopify Stores with real browsers. [Read more: Monitor your Shopify Store](https://testingbot.com/blog/shopify-store-monitoring) [![Generative AI test agent](https://testingbot.com/assets/blog/2025/testingbot-ai.webp) ](https://testingbot.com/blog/testingbot-ai-agent) Feb 18, 2025 ### [Generative AI test agent](https://testingbot.com/blog/testingbot-ai-agent) TestingBot has launched an AI Test Agent that will help you to create automated tests [Read more: Generative AI test agent](https://testingbot.com/blog/testingbot-ai-agent) [![Automated Testing with TestingBot during the Holiday Season](https://testingbot.com/assets/blog/2024/holidayseason.webp) ](https://testingbot.com/blog/testing-holiday-season) Nov 19, 2024 ### [Automated Testing with TestingBot during the Holiday Season](https://testingbot.com/blog/testing-holiday-season) Augment your testing this holiday season with TestingBot. [Read more: Automated Testing with TestingBot during the Holiday Season](https://testingbot.com/blog/testing-holiday-season) [![Selenium 20th birthday](https://testingbot.com/assets/blog/2024/selenium20.webp) ](https://testingbot.com/blog/selenium-20-years) Oct 29, 2024 ### [Selenium 20th birthday](https://testingbot.com/blog/selenium-20-years) Celebrate Selenium's 20th birthday. An overview of the years. [Read more: Selenium 20th birthday](https://testingbot.com/blog/selenium-20-years) ## Articles by Jochen Sep 1, 2026 ### [Parallel Testing: What Actually Scales](https://testingbot.com/resources/articles/parallel-testing) Why adding workers stops helping, how to shard so they are not idle, and what breaks the first time a suite runs concurrently. Aug 31, 2026 ### [Automated Regression Testing in CI](https://testingbot.com/resources/articles/automated-regression-testing) How to build a regression suite that stays fast and trustworthy in CI, what to leave out of it, and when to run which part. Aug 31, 2026 ### [Mobile App Test Automation at Scale](https://testingbot.com/resources/articles/mobile-app-test-automation) How to choose between Appium, Espresso, XCUITest and Maestro, and what changes when you run the suite on real devices at scale. Aug 31, 2026 ### [AI in Software Testing: What Works](https://testingbot.com/resources/articles/ai-in-software-testing) A category-by-category look at where AI genuinely helps a test suite, where it does not, and how to evaluate a tool before you buy it. Aug 31, 2026 ### [Playwright MCP for Test Automation](https://testingbot.com/resources/articles/playwright-mcp) What Playwright MCP does, how driving the accessibility tree differs from screenshot agents, and where it fits in a real test workflow. Sep 18, 2025 ### [Top Web Development Tools in 2025](https://testingbot.com/resources/articles/web-development-tools) The most popular web development tools in 2025, split into front-end, back-end and integrated, with what each one is actually good for. Jun 22, 2025 ### [Most popular Screen Resolutions](https://testingbot.com/resources/articles/common-screen-resolutions) The most common desktop, mobile and tablet screen resolutions in 2025, and what they mean for responsive design and the breakpoints you test. May 26, 2025 ### [Selenium Python Tutorial](https://testingbot.com/resources/articles/python-selenium-web-automation-test) Getting started with Selenium WebDriver in Python: a first test, locating elements, implicit and explicit waits, alerts and teardown. May 22, 2025 ### [How to Change Browser Settings](https://testingbot.com/resources/articles/how-to-change-browser-settings) Step-by-step instructions for changing proxy, cookie and cache settings in Chrome, Edge and Firefox, on both Windows and macOS. May 17, 2025 ### [What is isolation testing?](https://testingbot.com/resources/articles/isolation-test) Isolation Testing means verifying the functionality of a specific part of software on its own, free from outside influences. ## Get in touch with Jochen Questions about cross-browser testing, mobile automation, or working with TestingBot? Reach out. [Contact us](https://testingbot.com/contact/new) --- URL: https://testingbot.com/contact/new Talk to TestingBot # Contact TestingBot If you need support, have questions, or are interested in purchasing a plan, feel free to reach out, we'll get back to you promptly. Name Work email How can we help? The more context you share, the faster we can route your message to the right person. By sending this message you agree to our [privacy policy](https://testingbot.com/privacy). We use your details only to reply to your request. Send message Need a faster path? Skip the form for these common tasks. [Book a 15-minute demo Live walkthrough by an engineer](https://testingbot.com/demo) [Browse documentation Setup guides for every framework](https://testingbot.com/support) [Check system status Live uptime & incidents](https://status.testingbot.com) [Join our Slack community Get help from devs and our team](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) Other ways to reach us [Phone (US toll-free) +1 (855) 410-0024](tel:+18554100024) [Email info@testingbot.com](mailto:info@testingbot.com) Address TestingBotVlinderhof 18 9180 Moerbeke Belgium VAT BE 0842.864.474 ## Frequently Asked Questions Quick answers about contacting our team, demos, trials and security. How quickly will TestingBot respond to my message? Our team replies to most contact form messages within a few hours during European business hours, and within 24 hours otherwise. Paid customers get priority support around the clock. Should I contact sales or support? Use this form for sales, billing, partnership and general questions. For technical issues with a running test or your account, signed-in users can also reach support directly from the dashboard. Can I try TestingBot before talking to sales? Yes. You can sign up for a free account that includes 100 minutes of automated and manual testing on real browsers and devices, with no credit card required. Where is TestingBot hosted and is my data safe? TestingBot is built and hosted in Europe (Belgium). All test data, video recordings and logs stay on EU infrastructure. We are GDPR compliant and a CSA STAR Level 1 registered cloud provider. Can I schedule a live demo? Yes. Use the Book a Demo button on this page to pick a 15-minute slot. We will walk you through automated, manual and AI-driven testing on real browsers and devices. Do you offer enterprise plans, dedicated devices or on-premise? Yes. We offer enterprise plans with dedicated browsers and devices, single sign-on, custom contracts and a dedicated private cloud option. Mention your needs in the message and our enterprise team will reach out. --- URL: https://testingbot.com/careers # Careers Join our team in maintaining, optimizing and improving our testing cloud. ![Helping developers and teams test. Reliably and at scale.](https://testingbot.com/assets/about/bg-2e0825c48447a2dbdefc8bb7e4bfe93c3fa7d218def5e88141411eb056d2c813.jpg) ## Open Positions - [Technical Content Writer](https://testingbot.com#writer) ## Technical Content Writer TestingBot is looking for a technical content writer, with experience in Selenium, Cypress, Puppeteer and Playwright. You will be adding content to our [Blog](https://testingbot.com/blog) and [Resources](https://testingbot.com/resources) section. ### Responsibilities - Write articles, tutorials and courses about Selenium, Cypress, Puppeteer and Playwright. - Research interesting topics to write about, related to the online testing ecosystem. - Add code examples, screenshots and tips & tricks to the articles you write. ### Key Skills - English writer - Previous content writing experience - Faultless grammar and spelling - IT technical knowledge an advantage, especially if you have knowledge about either of these topics: Selenium, Cypress, Puppeteer or Playwright - Excellent attention to detail ### Benefits - Competitive salary - Flexible schedule - Work from home (remotely) Interested? Please send us a brief introduction and resume. [Apply now](https://testingbot.com/contact/new) --- URL: https://testingbot.com/terms # Terms of Service PLEASE READ CAREFULLY. THESE TERMS AND CONDITIONS ("THE AGREEMENT") IS A LEGALLY BINDING AGREEMENT BETWEEN YOU ("YOU" OR "YOUR") AND TESTINGBOT. ("TESTINGBOT", "WE" OR "OUR"). BY CHECKING THE APPROPRIATE BOX INDICATING THAT YOU AGREE DURING REGISTRATION OR OTHERWISE ACCESSING THE TESTINGBOT WEBSITE, SERVICES, SOFTWARE OR MATERIALS (COLLECTIVELY THE "SERVICE"), YOU ARE INDICATING THAT YOU HAVE READ THIS AGREEMENT, THAT YOU UNDERSTAND IT, AND THAT YOU CONSENT TO BE BOUND BY ALL OF ITS TERMS AND CONDITIONS. IF YOU DO NOT AGREE WITH ALL OF THE FOLLOWING TERMS AND CONDITIONS, PLEASE DO NOT CONTINUE THE REGISTRATION PROCESS OR USE THE SERVICE. IF YOU WISH TO USE TESTINGBOT AS AN EMPLOYEE, CONTRACTOR, OR AGENT OF A CORPORATION, PARTNERSHIP OR SIMILAR ENTITY, THEN YOU MUST BE AUTHORIZED TO SIGN FOR AND BIND THE ENTITY IN ORDER TO ACCEPT THE TERMS OF THIS AGREEMENT. THE LICENSES GRANTED AND SERVICES PROVIDED UNDER THIS AGREEMENT ARE EXPRESSLY CONDITIONED UPON ACCEPTANCE BY SUCH AUTHORIZED PERSONNEL. This End User License Agreement ("Agreement") is entered into by and between TestingBot and the customer (individual or entity) that has procured the TestingBot products or services for use as an end user ("Customer"). "Effective Date" means the date on which accepts the terms and conditions of this Agreement. TestingBot reserves the right, at its sole discretion, to modify or replace this Agreement, or change, suspend, or discontinue any TestingBot Product(s) or Service Descriptions (as those terms are defined below), including without limitation, the availability of any feature, database, or content) at any time by posting a notice on the TestingBot Product(s) or by sending you an email. It is your responsibility to check this Agreement periodically for changes. Your continued use of any TestingBot Product following the posting of any changes to this Agreement constitutes acceptance of those changes. ## 1. Provision of Products Welcome to TestingBot.com. We provide our customers with online software-as-a-service/platform-as-a-service products. We use Selenium, an open-sourced browser testing framework to test websites in multiple browsers in an automated way. We offer several plans according to the needs of the customer. Each plan is provided on a subscription basis for a set term designated at the time of purchase (each, as “Subscription Term”). Customer shall purchase and TestingBot shall provide the specific product and plan as specified at the time of purchase. Some elements of the Service are available to any User, while other elements ("Subscription Services") are only available to Users who have purchased a subscription from TestingBot according to these terms and conditions. TestingBot may at any time, in its sole discretion, change a freely available element of the Service to a Subscription Service, and vice versa. TestingBot may, it its sole discretion, offer Users a one-time, free trial period during which you may use Subscription Service(s) for free. The time period of any such free trial period may vary from time to time, and will be indicated at the time you register as a User. Please note that TestingBot does not provide any internet connectivity services, nor does TestingBot provide any computer equipment for Users. You are entirely responsible for securing and paying for any such services and equipment, and for ensuring that such services and equipment meet the minimum requirements for the use of the Services. General Restrictions; Acceptable Use. Customer shall not: (a) rent, lease, copy, provide access to or sublicense any TestingBot Product to a third party; (b) use any TestingBot Product to provide or incorporate any TestingBot Product into any product or service provided to a third party, (c) reverse engineer, decompile, disassemble, or otherwise seek to obtain the source code or non-public APIs to TestingBot Products, including the Video Playback, except to the extent such restriction is prohibited by applicable law (and only upon advance notice to TestingBot), (d) modify any TestingBot Product or any Documentation, or create any derivative product from any of the foregoing, (e) remove or obscure any proprietary or other notices contained in any TestingBot Product (including any reports or data printed from TestingBot Products) or (f) publicly disseminate information regarding the performance of TestingBot Products. ## 2. Accounts/Customer Data You may establish one (1) User account (the "Account") per email address you supply to TestingBot. You may not share the Account with anyone. You are liable for all uses of the Account. If you would like to have several accounts for co-workers or friends, please consider using a sub-account. Storage of Customer Data. TestingBot does not provide an archiving service. TestingBot agrees only that it shall not intentionally delete any Customer Data which is less than 30 days old from TestingBot Products prior to termination of Customer’s applicable Subscription except as directed by Customer or its Permitted Users. TestingBot expressly disclaims all other obligations with respect to storage. If specified at the time of purchase, TestingBot may charge Customer a fee for storing the Customer Data. ## 3. Registration During the registration process prior to using the Service, you may be requested to provide certain information (e.g. your birthdate, gender, zip code, name, e-mail address, country or territory of residence, and for paid services, credit card and billing information, such as your telephone number), and you agree to provide true, accurate, current and complete information about yourself as prompted by such process (such information being "Personal Information"). If you open an Account, and any of your Personal Information changes (e.g., new credit card number, expiration date, new address or email address, new phone number, etc.), you must update that information in your Account. ## 4. Subscription Services If you elect to subscribe to Subscription Services, the fee therefor will be the price specified at the time of purchase. You hereby agree to pay all charges incurred by the Account, including applicable taxes, in accordance with billing terms for access to the Service that are in effect at the time that the fee or charge becomes payable. Your right to access the Service is subject to any limits established by TestingBot or by your credit card issuer, billing service, or other payment methods authorized by TestingBot. TestingBot may, in its sole and absolute discretion, suspend or terminate your access to the Service and disable or delete the Account, if: (i) payment cannot be charged to your credit or debit card; or (ii) your charge is returned to TestingBot for any reason. If at any time you have a balance due on the Account, you agree that TestingBot can charge these unpaid fees to your credit card or debit them from your debit card, as applicable. You agree to reimburse TestingBot for all costs and expenses incurred by TestingBot in collecting payment due hereunder, including without limitation all bank or service charges, and any reasonable attorneys' fees. By subscribing to any Subscription Service you are representing to us that you are authorized to use the credit card you submit for payment. You agree to take such steps as are appropriate to ensure such compliance and will indemnify and hold TestingBot and TestingBot's affiliates and distribution and syndication partners harmless from any breach of this Agreement. Your subscription will last for the period chosen in the subscription process (the "Term"). Unless notifying us of your decision to terminate your subscription, your subscription will automatically be renewed at the end of each Term according to the then-current billing terms. TestingBot reserves the right to change the amount of, or basis for determining, any fees it charges, and to institute new fees, effective immediately upon notice to you. You may cancel your automatic membership renewal by contacting info@testingbot.com . Cancellation of your subscription will take effect upon the end of the then-current Term, at which time you will no longer have access to any Subscription Services. Our payment provider (Stripe or FastSpring) will automatically renew your paid subscription. Payment details such as your credit card number are not saved on this website. ## 5. TestingBot Materials The Service may require that you install plugins to report back test results to TestingBot. That software, and all new versions, updates, upgrades, bug fixes, modifications and derivative works that TestingBot may make available from time to time are referred to collectively as the "Software." All Software, designs, text, images, photographs, illustrations, audio clips, video clips, artwork, graphic materials, other copyrightable elements, and the selection and arrangements thereof, and all trademarks, service marks, trade dress, trade names, logos and designs and all intellectual property rights therein, provided by TestingBot via the Service (collectively the "Materials") are the property of TestingBot and/or its business partners, affiliates, assigns, licensors or other respective owners and are protected by intellectual property laws, including without limitation, U.S. and international patent, copyright and trademark laws. TestingBot may include in some Materials or in conjunction with your User Supplied Content third party advertisements that appear or are played when those Materials or User Supplied Content are viewed. You hereby consent to the display and playback of these advertisements in conjunction with your User Supplied Content. To the extent you elect to use or access TestingBot Materials or User-Supplied Content from other Users, you also consent to the display of advertisements in those Materials and that User Supplied Content. Content generated during testing like logs, screenshots and videos will stay available for 30 days. After this period, TestingBot will remove the content. ## 6. License from TestingBot Subject to all of the terms and conditions of this Agreement, TestingBot hereby grants you a personal, limited, non-exclusive, non-transferable, revocable right to (a) view or play back Materials on, your compatible computer or device. All rights not expressly granted to you are reserved. For the avoidance of doubt, your use of the Service and the Materials is under license and will terminate if this Agreement terminates, regardless of whether the relevant Service or Materials is free or you have paid for it; you will not obtain any ownership interest in any Materials through this Agreement or otherwise. ## 7. Usage Restrictions You agree not to use, download, reproduce, modify, create derivative works from, display, perform, publish, distribute, disseminate, transmit, broadcast or circulate any Materials to any third party, except as expressly authorized by TestingBot in connection with the Service. The source code and/or object code to the Materials contains confidential and trade secret information belonging to TestingBot and its licensors, and you shall at all times take reasonable steps to protect the confidentiality of such information. You further agree that you will not disassemble, decompile or reverse engineer the Materials (except to the extent that this restriction is expressly prohibited by law, and then only upon providing TestingBot with prior written notice and opportunity to respond). Any unauthorized or prohibited use may subject the offender to civil liability and criminal prosecution under applicable federal and state laws. As between you and TestingBot, TestingBot owns all elements of the Account. Except as specifically permitted and enabled within the Service, you may not transfer your Account, or any component of your User-Supplied Content to any other person. Through the use of the Service you may not impair or violate the rights of TestingBot or any other party with respect to trademarks, logos, or other intellectual property or rights. You may not display or perform Materials or User Supplied Content in a manner or in a context that is, in TestingBot's sole discretion, unlawful, misleading, defamatory, infringing, libelous, disparaging, obscene, an endorsement of any unrelated product or service, an unauthorized parody, or otherwise objectionable. ## 8. Code of Conduct You may not: (i) post any User Supplied Content that is illegal, abusive, deceptive, pornographic, obscene, defamatory, slanderous, offensive, or otherwise inappropriate or harmful to any person (including any minor) or entity; (ii) sell, rent, lease, or otherwise assign any rights to any Materials or Service to any third party; (iii) remove any proprietary notices or labels on the Materials or Service; (iv) use the Materials or Service in conjunction with any device or service designed to circumvent technological measures employed to control access to, or the rights in, any content protected by the copyright laws of any jurisdiction; (v) use the Materials or Service for any commercial or illegal purpose or to harm minors in any way; (vi) use the Materials or Service to invade the privacy of, or obtain personal information about, any User, or to obtain a list of Users, or to access or use any User's User Supplied Content in a manner not specifically authorized by such User; (vii) except as expressly permitted by TestingBot, copy, modify, erase or damage any Materials or information contained on any TestingBot or third party servers; (viii) use the Materials or Service to violate any law or legal right of any third party, including any publicity or privacy right, copyright, other intellectual property right, or any contractual, fiduciary or confidentiality obligation or duty, or to take any action that is harassing, libelous, defamatory, abusive, tortious, threatening, harmful or otherwise objectionable; (ix) use the Materials or Service to post or transmit any unsolicited advertising, campaigning or promotional materials or to send any Materials from an anonymous or false address; (x) access or use any password protected, secure or non-public areas of the Service except as specifically authorized in writing by TestingBot (unauthorized individuals attempting to access these areas of the Service may be subject to prosecution); (xi) distribute any software viruses or other computer code designed to interfere with the functionality of the Service or any computer systems, software or hardware; (xii) use any data mining, robots, or similar data gathering, spoofing, or extraction tools in connection with the Materials Service; (xiii) mislead other Users or third parties as to your identity or to the origin of a message or content; or (xiv) frame or link to the Service or Materials except as expressly permitted in writing by TestingBot. ## 9. Objectionable or Explicit Content TestingBot has the right, but not the obligation, to monitor, modify and/or remove any Materials and/or User Supplied Content in the Service. TestingBot may monitor, modify and remove any Materials and/or User Supplied Content in its sole discretion. TestingBot is not responsible to you for any content or materials constituting all or part of any Materials, any User Supplied Content, or any other aspect of the Service that you might find objectionable. By viewing the Service, you may be exposed to content that you consider offensive. You take sole responsibility for such exposure. ## 10. Privacy For information regarding TestingBot's use of information collected in connection with the Service, please refer to the [TestingBot Privacy Policy.](https://testingbot.com/privacy) ## 11. Active Participation; Storage and Bandwidth Limits If you do not have a currently active subscription to the Subscription Service, a condition of your access to the Service is your "Active Participation" in the Service. Active Participation is defined as accessing the Service, including displaying Materials or sending User Supplied Content through the Service, at least once every 180 days. If you do not maintain Active Participation, TestingBot may, with or without providing you with notice, terminate your Account (or any part thereof) and/or your use of the Service, and may remove and discard any content, including any of your User Supplied Content and any related Materials within the Service. Additionally, with or without providing you with notice, we may place a maximum limit on the amount of User Supplied Content that you are allowed to upload into the Service, or the maximum bandwidth that you may utilize each day. We may change these limits in our sole discretion upon providing you with notice. If you exceed these limits, we may refuse to accept and/or remove any User Supplied Content, or we may suspend certain functions of the Service for your Account. ## 12. Changes to the Agreement We reserve the right, at our sole discretion, to change, modify or otherwise alter these terms and conditions at any time. Such changes and/or modifications shall become effective immediately upon posting. Please review the terms and conditions periodically. If you do not agree to be bound by (or cannot comply with) the Agreement as amended, your only remedy is to cancel your TestingBot Account and to cease using the Service. You will be deemed to have accepted the Agreement as amended thirty (30) days after the amended agreement is posted if you continue to use the Service. All other terms of this Agreement will continue in effect. ## 13. Notices TestingBot may provide you with any notices under this Agreement by means of a posting on the Service, by e-mail, or by sending a message to you through the Service.    A. A physical or electronic signature of the owner (or person authorized to act on behalf of the owner) of an exclusive right that is allegedly infringed.    B. Specific identification of the copyrighted work claimed to have been infringed, or if multiple copyrighted works are covered by a single notification, a list of each copyrighted work claimed to have been infringed.    C. Information related to the work(s) reasonably sufficient for TestingBot to promptly locate the work (e.g. title of work, location within the Service, etc.).    D. Information reasonably sufficient to permit TestingBot to directly contact the complaining party, such as a complete name and address, telephone number, and an email address.    E. A statement that the complaining party has a good TestingBot belief that use of the work(s) in the manner complained of is not authorized by the copyright owner, its agent, or the law.    F. A statement requesting that TestingBot take a specific act with respect to the alleged infringement (e.g., removal, access restricted or disabled), and    G. A statement that the information in the notification is accurate, and under penalty of perjury, that the complaining party is authorized to act on behalf of the owner of an exclusive right that is allegedly infringed. ## 14. Trademarks Except as expressly set forth in this Agreement, all trademarks, service marks, trade names, domain names, slogans, logos, and other indicia of origin that appear on or in connection with the Service and Materials are the property of TestingBot and/or its affiliates and licensors. You may not copy, display or use any of these marks without prior written permission of the mark owner. ## 15. International You agree to abide by U.S and other applicable export control laws and not to transfer, by electronic transmission, the Service, or otherwise, any content or software subject to restrictions under such laws to a national destination or person prohibited under such laws. ## 16. Promotions and Advertising TestingBot and/or its business partners may present advertisements and promotional materials on or through the Service. Your participation in any TestingBot promotional event is subject to the terms and conditions associated with that event. Your correspondence or business dealings with, or participation in promotions of, any third-party advertisers on or through the Service, including payment and delivery of related goods or services, and any other terms, conditions, warranties or representations associated with such dealings, are solely between you and such third-party. You agree that neither TestingBot nor its business partners shall be responsible or liable for any loss or damage of any sort incurred as the result of any such dealings or as the result of the presence of such third parties on the Service. ## 17. The Website and Links to Third Party Sites The Website means this web site, as accessed by any of the following domain names: testingbot.com, \*.testingbot.com and any web site owned or controlled by TestingBot (except as otherwise provided on such other site). However, the Website does not refer to any other web site linked to or from the Website, or to any third party site on which the Service may appear. The Service may present links to retailers and/or other third-party websites not owned or operated by TestingBot. Neither TestingBot nor its business partners are responsible for the availability of these outside sites or their contents. You understand and agree that neither TestingBot nor its business partners are responsible or liable, directly or indirectly, for any damage or loss caused or alleged to be caused by or in connection with your use of or reliance on any content of any such site or goods or services available through any such site or from any such advertiser. You should direct any concerns regarding these third-party sites to those sites' administrators. ## 18. Modifications to Service and Materials TestingBot may, without liability or obligation, in its sole discretion and at any time: (i) make modifications to, or discontinue, any Materials and/or the Service, including deleting any Downloaded Materials we may have placed on your computer, with or without providing you with notice (except as we may expressly agree to provide you with notice elsewhere in this Agreement); and/or (ii) require you to use modified Materials and/or Service (e.g. Software updates) in order to have continued access to the Service and Materials. The modifications may result in certain Materials being unavailable, or incompatible with certain devices. Other Users may similarly delete or cease offering their User Supplied Materials at any time without notice. Upon TestingBot giving you notice, you shall promptly cease the use of, and destroy, all copies of any specified Materials, or otherwise modify your usage as directed by TestingBot. You agree that neither TestingBot nor the TestingBot Affiliates shall be liable to you or to any third party for any modification, suspension or discontinuance of the Service or Materials. ## 19. Remedies You understand and agree that any unauthorized use of the Service or the Materials would result in irreparable injury to TestingBot and/or the TestingBot Affiliates for which money damages would be inadequate, and in such event TestingBot and the TestingBot Affiliates, shall have the right, in addition to other remedies available at law and in equity, to immediate injunctive relief against you. Nothing contained in this paragraph or elsewhere in this Agreement shall be construed to limit remedies or relief available pursuant to statutory or other claims that TestingBot and the TestingBot Affiliates may have under separate legal authority, including but not limited to, any claim for intellectual property infringement. You understand and agree that your cancellation of your Account is your sole right and remedy with respect to any dispute with TestingBot. You can cancel your Account by delivering notice to TestingBot at info@testingbot.com. ## 20. Indemnity You agree to indemnify, and hold harmless, TestingBot, the TestingBot Affiliates, and each of their respective officers, directors, employees, agents, licensors, representatives, and third party providers from and against all claims, actions, demands, causes of action, and other proceedings, and all losses, expenses, fees, fines, damages and costs, including reasonable attorneys' fees and court costs, arising out of or relating to your breach of this Agreement or improper use of the Service or Materials. TestingBot reserves the right to assume, at its sole expense, the exclusive defense and control of any matter subject to indemnification by you, in which event you will fully cooperate with TestingBot in asserting any available defenses. ## 21. Suspension or Termination TestingBot may, in its sole discretion, terminate this Agreement or suspend your Account at any time with or without notice to you in the event that you breach (or TestingBot reasonably suspects that you have breached) any provision of this Agreement. If TestingBot terminates this Agreement, or suspends your Account for any of the reasons set forth in this paragraph, TestingBot shall have no liability or responsibility to you, and you agree that you forfeit any amounts you have paid TestingBot. TestingBot may terminate this Agreement and your Account for any reason or no reason at the end of the then-current Term by providing notice to you. You may cancel your Account (including Subscription Services) at any time by contacting info@testingbot.com. All fees paid and charges made prior to termination as provided herein are nonrefundable, including any advance charge or payment for the subscription term during which you terminate your subscription. Termination of your subscription shall not relieve you of any obligations to pay accrued charges. Upon termination of this Agreement for any reason, all of your rights to use the Service and materials shall immediately terminate, and you shall immediately cease using and destroy all Materials. ## 22. Disclaimers You understand and agree that your use of the Service and the Materials is at your own sole risk. TestingBot is not responsible or liable for any infections or contamination of your system, or delays, inaccuracies, errors, or omissions arising out of your use of the Service or Materials. The entire risk as to the quality, accuracy, adequacy, completeness, correctness and validity of any Materials rests with you. THE SERVICE AND THE MATERIALS ARE PROVIDED "AS IS" AND "AS AVAILABLE," AND WITHOUT WARRANTY BY TESTINGBOT AND THE TESTINGBOT AFFILIATES, AND, TO THE MAXIMUM EXTENT ALLOWED BY APPLICABLE LAW, TESTINGBOT AND THE TESTINGBOT AFFILIATES EXPRESSLY DISCLAIM ALL WARRANTIES, EXPRESS AND IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE AND ANY WARRANTY OF NONINFRINGEMENT. THERE IS NO WARRANTY THAT THE SERVICE OR THE MATERIALS WILL MEET YOUR REQUIREMENTS, OR THAT YOUR ACCESS TO THE SAME WILL BE UNINTERRUPTED OR ERROR-FREE. TESTINGBOT AND THE TestingBot AFFILIATES DO NOT WARRANT, GUARANTEE, OR MAKE ANY REPRESENTATIONS REGARDING THE USE OR THE RESULTS OF THE USE OF THE SERVICE OR THE MATERIALS WITH RESPECT TO PERFORMANCE, ACCURACY, RELIABILITY, SECURITY CAPABILITY, CURRENTNESS OR OTHERWISE. YOU WILL NOT HOLD TESTINGBOT AND/OR THE TESTINGBOT AFFILIATES RESPONSIBLE FOR ANY DAMAGES THAT RESULT FROM YOU ACCESSING THE SERVICE OR USING THE SERVICE AND/OR THE MATERIALS. NO ORAL OR WRITTEN INFORMATION OR ADVICE GIVEN BY ANY PERSON SHALL CREATE A WARRANTY IN ANY WAY WHATSOEVER RELATING TO TESTINGBOT AND/OR THE TESTINGBOT AFFILIATES. UNDER NO CIRCUMSTANCES SHALL TESTINGBOT AND/OR THE TESTINGBOT AFFILIATES BE LIABLE FOR ANY UNAUTHORIZED USE OF THE SERVICE AND/OR THE MATERIALS. ## 23. LIMITATION OF LIABILITY TO THE FULLEST EXTENT PERMISSIBLE PURSUANT TO APPLICABLE LAW, UNDER NO CIRCUMSTANCES SHALL TESTINGBOT AND/OR THE TESTINGBOT AFFILIATES BE LIABLE TO YOU FOR ANY CONSEQUENTIAL, INCIDENTAL, INDIRECT, SPECIAL OR SIMILAR DAMAGES (INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF BUSINESS PROFITS, BUSINESS INTERRUPTION, LOSS OF BUSINESS INFORMATION, AND THE LIKE) ARISING OUT OF THE USE OR INABILITY TO USE THE SERVICE AND/OR THE MATERIALS, EVEN IF TESTINGBOT AND/OR THE TESTINGBOT AFFILIATES, HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. BECAUSE SOME JURISDICTIONS DO NOT ALLOW THE EXCLUSION OR LIMITATION OF LIABILITY FOR CONSEQUENTIAL OR INCIDENTAL DAMAGES, THE ABOVE LIMITATION MAY NOT APPLY TO YOU. TO THE EXTENT THAT IN A PARTICULAR CIRCUMSTANCE ANY DISCLAIMER OR LIMITATION ON DAMAGES OR LIABILITY SET FORTH HEREIN IS PROHIBITED BY APPLICABLE LAW, THEN, INSTEAD OF THE PROVISIONS HEREOF IN SUCH PARTICULAR CIRCUMSTANCE, TESTINGBOT AND/OR THE TESTINGBOT AFFILIATES, SHALL BE ENTITLED TO THE MAXIMUM DISCLAIMERS AND/OR LIMITATIONS ON DAMAGES AND LIABILITY AVAILABLE AT LAW OR IN EQUITY BY SUCH APPLICABLE LAW IN SUCH PARTICULAR CIRCUMSTANCE, AND IN NO EVENT SHALL SUCH DAMAGES OR LIABILITY EXCEED TWENTY FIVE DOLLARS ($25). Notwithstanding the foregoing, in no event shall TestingBot's liability to you for any and all claims, damages, losses, and causes of action (whether in contract, tort or otherwise) exceed the amount paid by you, if any, for accessing the Service. ## 24. Claims and Arbitration Any action hereunder by a User must be brought, if at all, within one (1) year from the accrual of the cause of action. You agree that the all disputes and claims arising out of, or relating to, this Agreement, the Service or Materials, your use of the Service or Materials (including the arbitrability of any claim or dispute and the enforceability of this paragraph), or to any other alleged act or omission by you or TestingBot, shall be determined exclusively by final and binding arbitration. Provided however, that you agree that the terms of this paragraph do not apply to any breach or alleged breach of the ownership or license rights of TestingBot or any of its licensors with respect to any Materials and/or Service, or your (or any third party's) misuse of the Materials and/or Service, and that, in the event of any such breach of alleged breach, TestingBot and its licensors will be irreparably harmed and entitled to equitable and injunctive relief in addition to all other remedies provided by this Agreement or available at law. Except as expressly set forth above in this paragraph, you and TestingBot may litigate in court only to compel arbitration under this Agreement or to confirm, modify, vacate or enter judgment on the award rendered by the arbitrators. To the extent that you have breached or have indicated your intention to breach this Agreement in any manner which violates or may violate TestingBot's or any of its licensor's intellectual property rights, or may cause continuing or irreparable harm to TestingBot (including, but not limited to, any breach that may impact TestingBot's or its licensor's intellectual property rights, or a breach by reverse engineering), TestingBot may seek injunctive relief, or any other appropriate relief, in any court of competent jurisdiction. To the fullest extent permitted by applicable law: no arbitration under this Agreement shall be joined to an arbitration involving any other current or former licensee of TestingBot, whether through class arbitration proceedings or otherwise; no finding or stipulation of fact in any other arbitration, judicial or similar proceeding may be given preclusive or collateral estoppel effect in any arbitration hereunder (unless determined in another proceeding between you and TestingBot); and no conclusion of law in any other arbitration may be given any weight in any arbitration hereunder (unless determined in another proceeding between you and TestingBot). ## 25. Miscellaneous This Agreement and any other terms or documents referred to herein represent your entire agreement with TestingBot with respect to your use of the Service and Materials. As used in this Agreement the term "licensor" shall not refer to the person at that time entering into the Agreement. You understand and agree that, except as expressly set forth herein, this Agreement is not intended to confer and does not confer any rights or remedies upon any person other than the parties to this Agreement. If any part of this Agreement is held invalid or unenforceable, that portion shall be construed in a manner consistent with applicable law to reflect, as nearly as possible, the original intentions of the parties, and the remaining portions shall remain in full force and effect. TestingBot's failure to pursue any available claim or defense pursuant to this Agreement or otherwise will not be a waiver of such claim or defense. The headings used in this Agreement are for convenience only, and will have no effect on the interpretation or legal enforceability of the terms herein. This Agreement is personal to you and may not be transferred, assigned or delegated to anyone. Any attempt by you to assign, transfer or delegate this Agreement shall be null and void. Last updated: May 9, 2017 --- URL: https://testingbot.com/privacy # Privacy Policy TestingBot knows that you care how information about you is used and shared, and we appreciate your trust that we will do so carefully and sensibly. By visiting testingbot.com, you are accepting the practices described in this Privacy Policy. TestingBot knows that you care how information about you is used and shared, and we appreciate your trust that we will do so carefully and sensibly. We process personal information contained in Customer Data on behalf of our customers under the terms and conditions of our [TestingBot Customer Data Processing Addendum](https://trust.testingbot.com). - [What's your policy about Cookies?](https://testingbot.com#cookies) - [What personal information does TestingBot gather?](https://testingbot.com#personal) - [Which information do you share?](https://testingbot.com#information) - [What Data Subject Rights Do I Have?](https://testingbot.com#rights) - [How Long Do We Retain Your Personal Data?](https://testingbot.com#duration) - [Is my information safe on this website?](https://testingbot.com#secure) - [Which information can I access?](https://testingbot.com#access) - [Are children allowed to use TestingBot?](https://testingbot.com#children) - [How can I remove all my information from TestingBot?](https://testingbot.com#delete) - [Who is the Data Controller?](https://testingbot.com#controller) - [What Are Our Legal Bases for Processing Personal Data?](https://testingbot.com#legal) - [Conditions of use, notices, and revisions](https://testingbot.com#changes) - [Implemented Technologies](https://testingbot.com#technologies) ## What's your policy about Cookies? Please consult our [Cookie Policy](https://testingbot.com/cookie-policy). ## What personal information does TestingBot gather? The information we learn from customers helps us personalize and continually improve your experience at testingbot.com. During signup, you are asked for your first name and last name, together with an e-mail address. When upgrading your plan, you may be asked for a telephone number and address by our billing provider. Information from Other Sources: We might receive information about you from other sources and add it to our account information. **Fraud prevention at signup:** when an account is created, we record technical characteristics of the request so we can detect fraudulent signups and abuse of free trial credits. This covers the IP address and derived network information (country, network operator), the browser user agent, and a device identifier computed locally in your browser from its configuration. We do not use this information for advertising, we do not share it with third parties, and it is not used to track you across other websites. It is retained with your account and deleted when your account is deleted. You are not required to provide any personal data to us, but if you do not provide any personal data to us, you may not be able to use certain features of our Services... ## Which information do you share? - **Subcontractors, sub-processors and service providers:** We share your firstname, lastname and email address with our payment providers (Stripe Inc. and Bright Market, LLC (FastSpring)) and with our customer support platform (Intercom). The complete, current list of sub-processors, including what each receives, is maintained at [trust.testingbot.com/sub-processor-updates](https://trust.testingbot.com/sub-processor-updates). - A third party, in order to enforce or defend our rights, or to address financial or reputational risks. - A third party to respond to requests relating to a criminal investigation or alleged or suspected illegal activity. - Other recipients where we are authorised or required by law to do so. - A purchaser or prospective purchaser of all or part of our assets or our business... ## What Data Subject Rights Do I Have? - **Right of access:** You have the right to obtain from us confirmation as to whether your personal data is being processed... - **Right to rectification:** You have the right to obtain from us the rectification of inaccurate personal data... - **Right to erasure:** You have the right to ask us to erase your personal data... - **Right to restriction of processing:** You have the right to request restriction of processing... - **Right to data portability:** You have the right to receive your personal data in a structured, commonly used... - **Right to object:** You have the right to object, on grounds relating to your particular situation... - **Right to Submit Complaints:** You have a right to lodge a complaint with a supervisory authority. ## How Long Do We Retain Your Personal Data? If you register for an account on our Services, we retain your personal data for as long as you have an account with us... ## Is my information safe on this website? TestingBot uses SSL (HTTPs) on all parts of its service/website. SSL encrypts information between you and our website. Payment data is handled by our payment providers (Stripe Inc. and Bright Market, LLC (FastSpring)); PayPal payments are processed through Stripe Checkout. Full card numbers never reach TestingBot servers; we store the last 4 digits only. All sensitive data is encrypted in our database and in our log files. Passwords and security tokens are stripped before they are logged into our system. We will notify serious data breaches... We restrict access to Account-Related Information and other Personal Information... ## Are children allowed to use TestingBot? We do not sell products for purchase by children. If you are under 18, you may use TestingBot only with the involvement of a parent or guardian. ## Which information can I access? You can access and modify your personal information in our member area. ## How can I remove all my information from TestingBot? Please log into our member area, click "Account" and click the "Remove account" button. ## Who is the Data Controller? If you are using our Services, the data controller is Norento BV (trading as TestingBot), Vlinderhof 18, 9180 Moerbeke, Belgium, +1 (855) 410-0024... ## What Are Our Legal Bases for Processing Personal Data? - **Contract Performance:** We process the personal data of users of our Services as necessary to perform our contractual obligations... - **Legitimate Interests:** To provide users with a good user experience, to maintain and secure our Services... - **Legal Obligations:** If we are subject to a lawful access request, engaged in a legal proceeding... - **Consent:** If we are required to obtain your consent to send you marketing communications... - **Vital Interests:** In extenuating circumstances, we may need to process your personal data to protect the vital interests... ## Conditions of use, notices, and revisions If you choose to visit testingbot.com, your visit and any dispute over privacy is subject to this Notice... We may periodically alter this policy and our Terms of service. Please check these pages frequently to see the most recent changes. **Last updated: August 5, 2026** (described the signup signals we record for fraud prevention, including local device identification) ## Implemented Technologies If you have any questions, complaints or remarks about this Privacy Policy, you can contact our Data Protection Officer at [info@testingbot.com](mailto:info@testingbot.com). TestingBot Attn: Data Protection Officer [info@testingbot.com](mailto:info@testingbot.com) Telephone number: +1 (855) 410-0024 [Contact TestingBot](https://testingbot.com/contact/new) --- URL: https://testingbot.com/cookie-policy # Cookie Policy TestingBot uses cookies to improve your user experience. ## What's a cookie? A cookie is a small file which is placed on your computer to remember your preferences. You can read more about cookies on [AllAboutCookies](https://allaboutcookies.org). ## How are you using cookies? When you first visit our website, our partner Google Analytics will set a cookie to track which pages you visit. Next to that, we will set a cookie to determine if you're logged in or not (session cookie). Once you log out, this cookie is destroyed. We offer online chat as a way to help you with your questions. This 3rd party provider also sets cookies to enhance your user experience. ## Cookie List Below is the list of cookies we use on TestingBot: ### Strictly Necessary Cookies These cookies are necessary for the website to function and cannot be switched off in our systems... #### Cookies used | Cookie Name | Description | Expiration | | --- | --- | --- | | \_testingbot\_gsession | Keep track of whether you're signed in on TestingBot | Until you close your browser | | remember\_user\_token | If you enabled "Remember me" during login, we will use this cookie to remember your session. | 14 days | | \_\_stripe\_mid | Used by Stripe. More info on [Stripe's Cookie Policy](https://stripe.com/cookies-policy/legal). | 1 year | ### Performance Cookies These cookies allow us to count visits and traffic sources so we can measure and improve the performance of our site... #### Cookies used | Cookie Name | Description | Expiration | | --- | --- | --- | | \_gid | Used by Google Analytics. [More info](https://www.en.advertisercommunity.com/t5/Google-Analytics-Code/Information-about-a-cookie-of-Google-Analytics-named-gid/td-p/1127468). | 30 days | | \_ga | Used by Google Analytics to identify unique sessions. | 30 days | ### Targeting Cookies These cookies may be set through our site by our advertising partners... #### Cookies used | Domain | Cookies | | --- | --- | | youtube.com | - SID - HSID - demographics - VISITOR\_INFO1\_LIVE - PREF - APISID - SSID - LOGIN\_INFO - YSC - SAPISID | | google.com | - APISID - SSID - NID - PREF - SID - SAPISID - HSID | ### Social Media Cookies These cookies are set by a range of social media services that we have added to the site... #### Cookies used | Cookie Name | Description | Expiration | | --- | --- | --- | | intercom-id-[app\_id] | Anonymous visitor identifier for the online chat functionality. | 9 months | | intercom-session-[app\_id] | Keeps track of chat sessions. Gives users access to their previous conversations and identifies them in the Messenger. | 1 week | | intercom-device-id-[app\_id] | Identifier for each unique device that interacts with the Messenger. | 9 months | This policy was last updated on October 8, 2025. Please see our [Terms & Conditions](https://testingbot.com/terms) and our [Privacy Policy](https://testingbot.com/privacy) for additional information. --- URL: https://testingbot.com/features/smart-tv-testing # Online Test Automation On Smart TVs OTT test automation on real living room hardware: Apple TV 4K running tvOS, hosted in our cloud. Manual and automated Apple TV app testing on real tvOS devices, straight from your browser. [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) ![Online Test Automation On Smart TVs](https://testingbot.com/assets/hero/smarttv-5af19a1f001b85050d5408ac75376cb652d49188245a452b9e7393e20762fd4b.svg) Trusted by some of the world's most innovative companies ## Automated Testing for Smart TV apps Run Appium tests on TV apps running on real Apple TV 4K hardware with tvOS. Get access to raw video and audio generated from the Smart TV device. Test DRM-protected content, by accessing a device via manual or automated testing. [Read SmartTV Testing Documentation](https://testingbot.com/support/app-automate/smart-tv) ### Automated Testing with Appium Upload your Smart TV app and run automated tests with Appium on the remote device. ### Connect to a remote TV device Access live video and audio generated from the device, through your own browser and control the remote as if it was in your hand. ### Device Coverage Test on real Apple TV 4K hardware running tvOS. ## Frequently Asked Questions ### What competitive edge does TestingBot offer with Smart TV testing, compared to its peers? TestingBot offers both manual and automated testing of apps on Smart TVs. View DRM-protected content, hear live audio of the device under test and manually send remote control (IR signals) to the device. ### How can I test Smart TV applications on a remote Smart TV device? Simply upload your app to TestingBot Storage and pick a device to test on. TestingBot provides a high quality video and audio stream of the device. ### Can I test Apple TV apps manually as well as with Appium? Yes. Manually control a real Apple TV 4K through your own browser, including sending remote control signals, and run automated Appium tests against the same device. ### Which Apple TV devices and tvOS versions can I test on? Testing runs on real Apple TV 4K hardware with tvOS 17.4, hosted in our European datacenter. Set deviceName to Apple TV 4K and platformName to tvOS in your capabilities. We do not currently offer Android TV, Fire TV or Roku. ### How do I run Apple TV automation testing with Appium? Upload your tvOS app to TestingBot Storage, then point Appium at our hub with platformName tvOS and the app capability set to your uploaded build. Apple TV automation testing then behaves like any other Appium session, with video, logs and screenshots captured for every run. ### Sign up for a Free Trial Start testing your apps with TestingBot. [Start a free trial](https://testingbot.com/users/sign_up) --- URL: https://testingbot.com/features/manual-browser-testing Live cross-browser testing # Cross browser testing online, on real browsers Take control of any real Chrome, Safari, Firefox, Edge or Internet Explorer running on real macOS, Windows or Linux. No VM, no plugin, no install. Open a tab, click a browser, start testing in seconds. [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) - **Combinations:** 6100+ - **Platforms:** 5 - **Uptime:** 99.9% _A 2 by 2 grid of remote machines running in parallel in the TestingBot cloud: macOS with Safari, Windows 11 with Edge, Linux with Firefox, and Windows 7 with Internet Explorer 11. Each is opened in a browser tab, each shows DevTools inspecting the same web page._ Trusted by some of the world's most innovative companies 30-second tour ## Why teams pick TestingBot for live browser testing Real, never emulated Every session runs on a real macOS, Windows or Linux VM with the real browser binary. No headless shims, no fake Internet Explorer. The DOM, the JS engine and the cipher suite are exactly what your customer sees. From IE 8 to the latest beta Chrome, Safari, Firefox, Edge, Opera and every Internet Explorer back to IE 8. macOS Tahoe through Yosemite, Windows 11 through Windows 7, and Linux. The full matrix, always patched. Zero install, zero setup No browser extensions, no Java applet, no agent on your machine. Click a browser, our cloud streams a live, interactive desktop into the tab you already have open. Capabilities ## Everything a real browser session gives you A live cross-browser test should feel like sitting at the customer's machine. Native dev tools, real input, real audio, real network conditions. - ### Native DevTools Open the same DevTools your developers use locally. Chrome DevTools, Safari Web Inspector, Firefox DevTools and the F12 tools in Edge and IE, fully interactive over the live stream. - ### Streamed audio Hear video playback, autoplay policies, Web Audio output and HTML5 media events the way the customer would. Audio streams from the remote VM straight into your tab. - ### Real keyboard and mouse Type, scroll, drag, right-click and use modifier keys. The remote browser sees the same input events as a local one, so hover menus and keyboard traps behave normally. - ### Two-way clipboard Paste credentials, JSON fixtures or long URLs from your machine into the remote browser. No plugin to install. Copy data back out the same way. - ### Any resolution, on the fly Switch between 1280x720, 1920x1080, 4K and custom resolutions mid-session to validate responsive breakpoints without restarting the browser. - ### Geolocation testing Run a session that exits from the United States, the EU, the UK, Australia, Japan or other regions. Validate currency, language detection, GDPR banners and CDN behaviour. - ### Network throttling Simulate Edge, 2G, 3G, 4G and offline conditions. Catch perceived-performance bugs before they hit users on a train or in an elevator. - ### Mobile browsers Live sessions on real iPhones, iPads and Android phones from Samsung, Google, Xiaomi and OnePlus. Mobile Safari, Chrome on Android, Samsung Internet, no emulator in sight. - ### Screenshots & video Every session records a full video and lets you snap, annotate and share screenshots in two clicks. Send bugs straight to Jira, Slack or GitHub. - ### Beta and dev builds Reproduce bugs on Chrome Beta, Firefox Nightly, Safari Technology Preview and Edge Dev within hours of upstream release. Browser and OS coverage ## A real cross-browser matrix, not a wish list Real Chrome, Safari, Firefox, Edge, IE and Opera on real macOS, Windows and Linux. Live, on demand, with versions going back as far as IE 8 for legacy regression coverage. ### Desktop browsers - ![Chrome](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg "Chrome") Chrome - ![Safari](https://testingbot.com/assets/environments/svg/safari-e4837bb5f9a9729fef843af4180372e975c62587808b8ab479f615f3e240e858.svg "Safari") Safari - ![Firefox](https://testingbot.com/assets/environments/svg/firefox-90dce8de549728e54bd3209cfacb9f1a868c2c8e70d243eb5b930d69cbb2fb1e.svg "Firefox") Firefox - ![Edge](https://testingbot.com/assets/environments/svg/edge-b639cac0c7a843271a0d3a6ccf894cc296064b63b2cd2920209bc5d030f3c70b.svg "Edge") Edge - ![Opera](https://testingbot.com/assets/environments/svg/opera-fe6e1d716472033125a969dfe6489ab45f0eded91e9260292d6c4ce9b682a330.svg "Opera") Opera - ![IE 11](https://testingbot.com/assets/environments/svg/ie-075955285f8005808f3d26947aace877f96e5843a33bd4d6e997709d3a294451.svg "IE 11") IE 11 - ![IE 10](https://testingbot.com/assets/environments/svg/ie-075955285f8005808f3d26947aace877f96e5843a33bd4d6e997709d3a294451.svg "IE 10") IE 10 - ![IE 9](https://testingbot.com/assets/environments/svg/ie-075955285f8005808f3d26947aace877f96e5843a33bd4d6e997709d3a294451.svg "IE 9") IE 9 - ![IE 8](https://testingbot.com/assets/environments/svg/ie-075955285f8005808f3d26947aace877f96e5843a33bd4d6e997709d3a294451.svg "IE 8") IE 8 - ![Chrome Beta](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg "Chrome Beta") Chrome Beta Updated within hours of every Chromium and Firefox stable release. Beta and Nightly channels available on request. [View the full browser matrix](https://testingbot.com/support/web-automate/browsers) ### Operating systems - ![macOS Golden Gate (27)](https://testingbot.com/assets/environments/goldengate_logo-c186d316ab997e287f19e935e295e6e6421e52c309ebcdb54b20adff396714d0.png "macOS Golden Gate (27)") macOS Golden Gate (27) - ![macOS Tahoe (26)](https://testingbot.com/assets/environments/tahoe_logo-57c43a6991568db006f6302de4cda5171c2e1d56c4d3c0d81ce39ce7ae908146.png "macOS Tahoe (26)") macOS Tahoe (26) - ![macOS Sequoia (15)](https://testingbot.com/assets/environments/sequoia_logo-44f79649144ee4151cccc4b7e5f154d15d9a7050e81617ce3a7b9be9a88de018.png "macOS Sequoia (15)") macOS Sequoia (15) - ![macOS Sonoma (14)](https://testingbot.com/assets/environments/sonoma_logo-aeace52519331cc0c5c6e235d35f7ecb42e73abcbe5377d7518cfae7679cf403.png "macOS Sonoma (14)") macOS Sonoma (14) - ![macOS Ventura (13)](https://testingbot.com/assets/environments/ventura_logo-2191cecf16bacb3874bc9b896158b59e375a8d0d79e47665a77188fba804a4ac.png "macOS Ventura (13)") macOS Ventura (13) - ![macOS Monterey (12)](https://testingbot.com/assets/environments/monterey_logo-ea303be67e778d42b57c65e88f1f7d2de9f4c7f9fc3845fc1f9e2000133a144c.png "macOS Monterey (12)") macOS Monterey (12) - ![macOS Big Sur (11)](https://testingbot.com/assets/environments/bigsur_logo-a12e9501ec8d615b2196728977e14aabf6e10961fc5fcfb51670f949e50f8364.png "macOS Big Sur (11)") macOS Big Sur (11) - ![Windows 11](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg "Windows 11") Windows 11 - ![Windows 10](https://testingbot.com/assets/environments/svg/windows10-f19b2b5055d9a272b1c8b35711154fc9cca1ae14442b48771e44680d28faffee.svg "Windows 10") Windows 10 - ![Windows 8.1](https://testingbot.com/assets/environments/svg/windows10-f19b2b5055d9a272b1c8b35711154fc9cca1ae14442b48771e44680d28faffee.svg "Windows 8.1") Windows 8.1 - ![Windows 7](https://testingbot.com/assets/environments/svg/windows-18aedb309faf40ff3447063f014404d18f2f97dee5deb28877d5f1e9787b2337.svg "Windows 7") Windows 7 - ![Linux](https://testingbot.com/assets/environments/svg/linux-c2988130428305e3fc734f852b695644aa5cc90e8df7ff6f0bc41cae70e4c43f.svg "Linux") Linux macOS Yosemite through macOS Tahoe, Windows 7 through 11, plus Linux images for headless-style regression. Real browsers vs. local VMs ## Why developers stop running their own browser farm Maintaining your own VirtualBox image of IE 11 on Windows 7 was already painful in 2018. Here is what teams get back when they move to TestingBot. | Capability | TestingBot | Local VM / BrowserStack snapshot | | --- | --- | --- | | Browser engine | Real Chrome, Safari, Firefox, Edge or IE running on a real OS | VM image that drifts out of date the moment you snapshot it | | Version coverage | IE 8 to the latest stable, plus Beta and Dev channels | Whatever you had time to download and license | | Setup | Sign in, click a browser, you are testing | VM tooling, license keys, network bridging, disk space | | Legal IE licensing | Covered by TestingBot's Microsoft licensing | You source and license each Windows image yourself | | Sharing a repro | One link, video, screenshots and logs included | Screen-record locally, upload to a chat tool, hope it plays | | Clean state | VM is wiped after every session, no carry-over cookies | Whatever the previous test left behind | Native browsers ## Real browsers on real macOS, Windows and Linux TestingBot does not ship a custom-patched Chrome or a "Chromium-based" Safari clone. Every browser runs on the real operating system it shipped for, with the real binary from Apple, Microsoft, Mozilla and Google. The CSS engine, the JS engine, the TLS stack, even font rendering match what your customer sees. - Real Chrome on real macOS, not Chromium on Linux - Real Safari on real macOS Tahoe through Yosemite - Real Internet Explorer 8, 9, 10 and 11 on the right Windows release - Real Edge on Windows, Linux and macOS ![Chrome](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg "Chrome") Chrome ![Safari](https://testingbot.com/assets/environments/svg/safari-e4837bb5f9a9729fef843af4180372e975c62587808b8ab479f615f3e240e858.svg "Safari") Safari ![Firefox](https://testingbot.com/assets/environments/svg/firefox-90dce8de549728e54bd3209cfacb9f1a868c2c8e70d243eb5b930d69cbb2fb1e.svg "Firefox") Firefox ![Edge](https://testingbot.com/assets/environments/svg/edge-b639cac0c7a843271a0d3a6ccf894cc296064b63b2cd2920209bc5d030f3c70b.svg "Edge") Edge ![IE 8 to 11](https://testingbot.com/assets/environments/svg/ie-075955285f8005808f3d26947aace877f96e5843a33bd4d6e997709d3a294451.svg "IE 8 to 11") IE 8 to 11 ![Opera](https://testingbot.com/assets/environments/svg/opera-fe6e1d716472033125a969dfe6489ab45f0eded91e9260292d6c4ce9b682a330.svg "Opera") Opera macOS Yosemite to Golden Gate Windows 7 to 11 Linux Ubuntu Mobile Safari iPhone, iPad Chrome for Android Google, Samsung, OnePlus Samsung Internet Galaxy S, A Firefox for Android Optional Real devices and simulators/emulators Tap, pinch, rotate, scroll Mobile browsers ## Real iPhones, iPads and Android phones from the same tab Switch from a desktop Chrome session to a real iPhone running Mobile Safari, or a real Galaxy running Chrome on Android, without changing tools. Tap, scroll, pinch-zoom, rotate, and inspect the page with the real device's Safari Web Inspector or Chrome DevTools. [iPhone testing](https://testingbot.com/test-on-iphone) [iPad testing](https://testingbot.com/test-on-ipad) [Galaxy testing](https://testingbot.com/test-on-galaxy) Live exit nodes 10+ countries United States United Kingdom Germany France Spain Belgium Australia China Japan Network throttling sits alongside geolocation: simulate Edge, 2G, 3G, 4G or offline in the same session. Geolocation testing ## Test from where your customers actually are Route a live session through an exit node in the United States, the UK, Germany, France, Japan, Australia or other regions. Validate currency, language detection, ad bidding, GDPR consent flows and edge CDN behaviour without flying anywhere. Secure tunnel ## Test staging and internal apps behind your firewall TestingBot Tunnel opens a secure, encrypted route from our cloud browsers to _your_ local server, staging environment or internal network. No VPN, no firewall changes, full control of the proxy and policies. Start the tunnel from your laptop or your CI runner. The session loads your staging URL exactly the way the user will see it in production, with cookies, auth and SSO intact. ![TestingBot secure tunnel for live browser testing](https://testingbot.com/assets/features/tunnel-diagram-poster-8c25e19e6fe4f7276a46a96b31a0ad0e07bb4f5bf98dc192d2b70897cfbb1a26.png) [Read more](https://testingbot.com/support/tunnel) Security ## Pristine VMs, wiped after every session Each live test gets its own isolated virtual machine. We do not share state between customers. The instant you close the session, the disk is destroyed and the next test starts on a clean image. TestingBot is GDPR compliant with EU-only hosting, and supports SAML SSO on enterprise plans. - One customer per VM, never co-tenanted - Datacenter-hosted, not residential proxies - Disk and memory wiped at session end 1 customer per VM Datacenter hosted Wiped after each session CSA STAR Level 1 GDPR compliant SAML SSO enterprise From any OS ## Need Safari from Windows? IE from a Mac? Just open a tab. Designers on Macs need to check Internet Explorer. Windows devs need to check Safari. Linux engineers need to check Edge. TestingBot gives every machine in your company access to every browser, without anyone installing a VM. The browser runs in our datacenter. You interact with it through a normal browser tab using HTML5. There is nothing to download. [Start free](https://testingbot.com/users/sign_up) Your machine ![macOS](https://testingbot.com/assets/environments/tahoe_logo-57c43a6991568db006f6302de4cda5171c2e1d56c4d3c0d81ce39ce7ae908146.png "macOS") macOS ![Windows](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg "Windows") Windows ![Linux](https://testingbot.com/assets/environments/svg/linux-c2988130428305e3fc734f852b695644aa5cc90e8df7ff6f0bc41cae70e4c43f.svg "Linux") Linux TestingBot cloud ![Chrome](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg "Chrome") Chrome ![Safari](https://testingbot.com/assets/environments/svg/safari-e4837bb5f9a9729fef843af4180372e975c62587808b8ab479f615f3e240e858.svg "Safari") Safari ![IE 11](https://testingbot.com/assets/environments/svg/ie-075955285f8005808f3d26947aace877f96e5843a33bd4d6e997709d3a294451.svg "IE 11") IE 11 Collaboration ## Report bugs without leaving the session Take and annotate screenshots, share the live session with a co-worker, and download the automatic video recording when the test ends. Each bug ships with the metadata an engineer actually needs: browser, OS, screen size, network conditions and console output. Jira Slack GitHub _A TestingBot bug report draft: annotated screenshot, capture strip, console output and network log, ready to share to Jira or Slack_ FAQ ## Frequently Asked Questions Do I need to install anything? No. Live browser sessions run entirely in your existing browser tab using HTML5. There is no agent, no Java applet, no browser extension and no Flash. Sign in, click a browser, and the live desktop streams straight into the tab. Can I hear audio from the remote browser? Yes. Audio from the remote VM is streamed over the same WebRTC connection as the video, so video playback, autoplay policies, Web Audio output and media events behave the way they do on a customer's machine. Can I change the screen resolution mid-test? Yes. Switch between common breakpoints (mobile, tablet, laptop, 1080p, 4K) or set a custom resolution at any point in the session. The remote browser resizes without restarting, so responsive design work does not lose context. Which browser versions are available? TestingBot covers Chrome, Safari, Firefox, Edge, Opera and Internet Explorer 8, 9, 10 and 11 across macOS Yosemite through Tahoe, Windows 7 through 11, and Linux. New stable releases land within hours of upstream availability. Beta, Dev and Nightly channels are available on request. Do I get the native developer tools? Yes. Chrome DevTools, Safari Web Inspector, Firefox DevTools and the F12 tools in Edge and IE are fully interactive over the live stream. You inspect, profile, edit and debug exactly the way you do locally. Can I test a site running on my laptop or staging? Yes. Start [TestingBot Tunnel](https://testingbot.com/support/tunnel) on your machine or in CI. The cloud browsers see your localhost, staging URL or internal-network site over an encrypted route, with no VPN or firewall change required. Does TestingBot record the session? Yes. Every live session is recorded as a downloadable video, plus screenshots, console logs and network HARs. Sharing a bug is one link, not a re-run. Is my data safe? Each session uses its own isolated VM that is destroyed after you disconnect. TestingBot is GDPR compliant with EU-only hosting. Enterprise plans add SAML SSO and dedicated infrastructure. Do I need a credit card to try it? No. The free trial does not require a credit card. Sign up with an email address and you can start a live browser session immediately. Is this free browser testing, and what does the trial include? Online browser testing on TestingBot starts on the free plan: no credit card, nothing to install, and a recent version of each browser. The 7-day full-access trial asks for a card, charges nothing during the trial, and unlocks every browser version, with live sessions, screenshots and the secure tunnel included. Paid plans start after that. ### More reasons to love TestingBot ### 24/7 support Got questions? Ask in the 24/7 in-app customer [chat support](https://testingbot.com#) or [send us an email](https://testingbot.com/contact/new). We are here to help you! TestingBot Support Hey 👋 How can we help? I need support ### Trusted by over 14,000 teams 6M+ Tests 120k+ Users 400+ Enterprises 106 Countries [Request a demo](https://testingbot.com/demo) ### Developer Documentation Everything you need to integrate TestingBot into your workflow. Browse guides, API references and SDKs for web and mobile testing. - [Quickstart tutorials for Selenium](https://testingbot.com/support/web-automate/selenium) - [Appium Examples](https://testingbot.com/support/app-automate/appium) - [Integrations](https://testingbot.com/support/integrations) - [List of browsers & devices](https://testingbot.com/support/web-automate/browsers) [Explore Documentation](https://testingbot.com/support) ### Enterprise-grade Security STAR Level 1 Certified GDPR Compliant SAML SSO Single Sign-On 99.9% Uptime SLA [Visit TestingBot Trust Center](https://trust.testingbot.com/) Ready in 60 seconds ## Start a live browser session in 60 seconds No credit card. No install. Real Chrome, Safari, Firefox, Edge and IE running in our cloud, ready when you are. No credit card No install, no VMs Cancel anytime [Start a free trial](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) --- URL: https://testingbot.com/features/manual-mobile-testing Live mobile app testing # Test Android and iOS apps online, on real devices Manually control [a physical mobile device](https://testingbot.com/support/app-automate/devices) in our cloud to test your mobile app. Upload an `.apk` or `.ipa`, or install straight from Google Play or the App Store, and test it online on a real Android phone or iPhone, directly from your own browser. Nothing to install locally, no Android SDK, no emulator setup. [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) - **Real devices:** 40 - **Platforms:** iOS & Android - **Uptime:** 99.9% ![Automated Mobile App Testing](https://testingbot.com/assets/hero/manual-app-testing-12fb4d3b5749d78078fa9929d74e109f8efe65f8af3522bdc23f4bfbab4089a6.webp) Trusted by some of the world's most innovative companies ## How to test an APK online You can test an APK online in your browser by uploading the .apk file to TestingBot and opening it on a physical Android phone in our datacenter. There is no Android SDK to install, no emulator image to download and nothing to configure on your own machine. The same four steps work for iOS - upload an .ipa and it opens on a real iPhone or iPad. 1. Sign in to the TestingBot dashboard and open **Live App Testing**. 2. Under **Select Source** , choose **Uploaded Apps** , then click **Upload App** and pick your build. You can also install straight from Google Play, the App Store or TestFlight instead of uploading a file. 3. Choose a real Android phone, iPhone or tablet from the device list. 4. The app installs and launches. Tap, swipe, type and rotate from your browser, exactly as you would on the phone in your hand. Supported formats are `.apk` and `.aab` for Android, `.ipa` for physical iOS devices and `.zip` for iOS simulators. Builds can also be pushed from a CI pipeline with a single call to the [Storage API](https://testingbot.com/support/api/storage#upload). [Start free](https://testingbot.com/users/sign_up) ## Reliable Device Farm Get instant access to a fast, reliable device infrastructure. ### 24/7 device availability Physical iOS and Android devices, available at any time of the day. ### Zero setup & Maintenance Focus on testing instead of worrying about setup and maintenance of a mobile device farm. ### Security Devices located in a datacenter in Europe. Each test runs on a secure and pristine device. ### Real Android and iOS devices Test across real iPhone, iPads, Google, OnePlus, Xiaomi and Samsung phones and tabs. ### Variety of devices Test on any device, flagship or budget, with support for the newest and legacy iOS and Android versions. ### Performance Our infrastructure is built for performance to eliminate test flakiness and latency. ## The devices you can test on TestingBot runs a lab of physical phones and tablets in a datacenter in Belgium. These are unmodified retail devices - nothing is jailbroken or rooted - and each one is wiped and re-imaged between sessions, so your build never inherits someone else's state. We currently keep **40 physical devices** online for app testing. Browse the full lineup of [iPhones and iPads](https://testingbot.com/test-on-iphone) and [Samsung Galaxy devices](https://testingbot.com/test-on-galaxy), or see [every device and OS version](https://testingbot.com/support/app-automate/devices) we keep online. ![Test an app online on Pixel 8](https://testingbot.com/assets/devices/160h/46-d0ca4c1d1985aefc27c52ba2a1c88748875c574d5358e53adcbdeaee4ed078bb.webp) Pixel 8 ![Test an app online on Pixel 11](https://testingbot.com/assets/devices/160h/45-b90994534fadde53be99a94521c6ea23a39fdf32ad277240be4e2613540b5019.webp) Pixel 11 ![Test an app online on iPhone 17 Pro](https://testingbot.com/assets/devices/160h/44-6b4402c6996d9b1620879bb1123c40a6b36abb064e65dca0a1b3f05286779e9c.webp) iPhone 17 Pro ![Test an app online on iPhone 17 Pro Max](https://testingbot.com/assets/devices/160h/43-59d7e30684332ed4bf25567cbc09ca942eac46d925ccde8352d3e85812480f7d.webp) iPhone 17 Pro Max ![Test an app online on iPhone 16 Pro](https://testingbot.com/assets/devices/160h/42-de3704079345026847b739a5b1cf5332595904e54ede66abd988a76fb1e6593c.webp) iPhone 16 Pro ![Test an app online on iPad (9th generation)](https://testingbot.com/assets/devices/160h/41-70f7d18afde7620063a49a76cbfd088f8b59c742f3a5f4dd4a513dd6ec5c676c.webp) iPad (9th generation) ![Test an app online on Galaxy S26](https://testingbot.com/assets/devices/160h/40-e917ed08e15b5941fed75855ac457dd1df46b471a2020f081c4d70cefb68f4e8.webp) Galaxy S26 ![Test an app online on Huawei P40](https://testingbot.com/assets/devices/160h/39-32a788c7e66a3da1c88eaaefefd37381f729e510f096b43906968b6907c44c57.webp) Huawei P40 ![Test an app online on iPhone Air](https://testingbot.com/assets/devices/160h/38-c00a59944eb2e410d375e3f5b39780043f72446bb33b0aea6a26a0c45f309ea1.webp) iPhone Air ![Test an app online on Pixel 10](https://testingbot.com/assets/devices/160h/37-738ed465eff0a4bcc33a5294f6f9f4d701fed317ce0e97e202ffc3f9dca7edd4.webp) Pixel 10 ![Test an app online on iPhone 17](https://testingbot.com/assets/devices/160h/36-06478e9d9935ed794bd3ec02c85ad3a8eafe1de3cc11debd15351c1bdc99005b.webp) iPhone 17 ![Test an app online on Pixel 9](https://testingbot.com/assets/devices/160h/35-abb9a661ebbfa84f18bd6985d3e80330ad43fe4ea7d408157c87ae0f560a5ca1.webp) Pixel 9 ## Take control of a device Interact with the remote mobile device, just as you would with a device in hand. Tap, scroll, zoom, swipe and more. ![Cut suite time from hours to minutes](https://testingbot.com/assets/features/control-device-114c527535e472dd402c03580fdb23b36ab5716387bc5448c5d6d4262219b655.jpg) ## Test native device features Access and modify device features to test your app in various real-world conditions. ### Network Simulation Test for scenarios like connectivity, fluctuating network & many more. ### Localization testing Test your apps across different timezones, languages and geolocations. ### Dark Mode Test your app's UI with a dark background to ensure a smooth user experience. ### App Orientation Test your app in the landscape or portrait mode. Switch between orientations to test your app's response. ### Audio Record and hear in realtime the audio generated on the device. ### Settings Control various settings on the device. ## Control device See realtime generated logs from the remote iOS/Android device. Rotate the device, change its GPS location, modify the system language and more. Inspect the logs for errors, crashes or stacktraces. [Start free](https://testingbot.com/users/sign_up) ![Private Device Cloud](https://testingbot.com/assets/features/manual-app-ui-87103cae76ed7710e8af70ad58a3ef37a36bf41aa29566580a5e36bcdef89cae.webp) ## Inspect your app Find locators to use in your Automated Tests. TestingBot highlights the area in your layout while you go over the different elements in the layout hierarchy. Easily locate XPath, XCUITest and Espresso locators which you can then use for Automated Testing. ![Private Device Cloud](https://testingbot.com/assets/features/inspector-f48a920e82606a3a941d6d899b27e5a85ca3e63b474155e36ab6cff7d6ed7571.webp) ## TestingBot Storage Upload your native mobile app, as an .apk or .ipa, to [TestingBot Storage](https://testingbot.com/support/api#upload) and start testing immediately. [Start free](https://testingbot.com/users/sign_up) ![Cut suite time from hours to minutes](https://testingbot.com/assets/features/tb-storage-a1d1845fbed2bac0012e225de7f847431af6ffdb9dd9b6233fd3145ef3681cb9.jpg) ## Your very own Mobile Devices Choose your own iOS and Android devices. We will purchase these devices, install and configure them in our secure datacenter. These devices will be only available to you, 24x7. Private devices can be customised to your needs. Have us install specific profiles and apps, or change specific settings of the device. You decide. [Learn more](https://testingbot.com/enterprise/private-device-cloud) ![Private Device Cloud](https://testingbot.com/assets/enterprise/datacenter-d4fc366e9a4e8a106ec41e6817e21adb2aeb366a397ac258fd29ba9cd4de72c4.jpg) Secure Tunnel ## Test Staged Websites Use the secure TestingBot Tunnel to easily test your staging and development environments across all browsers, ensuring seamless access behind firewalls or private networks. Test data and websites from behind your firewall on the TestingBot cloud. ![TestingBot Tunnel](https://testingbot.com/assets/features/tunnel-diagram-poster-8c25e19e6fe4f7276a46a96b31a0ad0e07bb4f5bf98dc192d2b70897cfbb1a26.png) [Read more](https://testingbot.com/support/tunnel) ## Debug the app while you are testing it A bug you hit by hand should not have to be reproduced twice. Every live session on TestingBot records what happened while you were poking at the app, so you can hand a colleague the evidence instead of a description. Device logs stream in real time. Android logcat and iOS syslog appear next to the device while you use it, so a crash shows its stacktrace the moment it happens. A screen recording of the whole session is kept with the test, alongside any screenshots you take. The Mobile Inspector shows the live layout hierarchy and hands you XPath, XCUITest and Espresso locators you can paste straight into an automated test. Change GPS location, system language, orientation and network conditions mid-session, then keep testing without restarting the device. ## Emulator or real device? Both are useful, and TestingBot gives you both. The honest answer is that they answer different questions. An Android emulator or iOS simulator is fast, cheap and fine for checking that a screen renders and a flow does not crash. It runs your app against a desktop CPU and a synthetic GPU, so it will not tell you how the app behaves on a mid-range phone with two years of wear, a throttled radio and a battery at 12%. A physical device tells you about the things that only exist in hardware - camera and sensor behaviour, biometrics, push notifications, real network handover, thermal throttling and the specific rendering quirks of a particular vendor's Android skin. Most teams we work with run emulators in CI on every commit and physical devices before a release goes out. If you want the longer version of this argument, we wrote it up on [real device testing](https://testingbot.com/real-device-testing). More on the Android side specifically: [running an Android emulator online](https://testingbot.com/android-emulator-online). ## Frequently Asked Questions ### Should I use a real device or an emulator? TestingBot provides both emulators/simulators, as well as physical devices. Emulators are good if you want to test basic functionality, but they do not provide the same experience as a real device. We recommend using both real devices and emulators, depending on your needs. ### Where are the devices located? TestingBot offers physical devices located in Belgium (Europe). ### Which apps can I test? You can test any app that is available in the Apple App Store or Google Play Store. You can also upload your own app (.ipa or .apk) and test it on a real device. ### Do I need the Android SDK to test an Android app online? No. The device runs in our datacenter and you drive it from a browser tab, so there is no Android SDK, no emulator image and no local toolchain to keep up to date. Uploading a build and opening it on a phone is the whole setup. ### Can I test an APK online for free? You can start without paying anything. Signing up is free and needs no credit card or local install. Full access to every real iOS and Android device comes with the 7-day trial, which asks for a card but charges nothing during the trial. Paid plans start after that. ### Can I test an iOS app on a real iPhone from my browser? Yes. Upload an `.ipa`, or install the app from the App Store or TestFlight, and it opens on a physical iPhone or iPad in our datacenter. You control it from a browser tab - no Mac, no Xcode and no provisioning of your own hardware. ### Which file formats can I upload? For Android, upload an `.apk` or an `.aab`. For iOS, upload an `.ipa` for a physical iPhone or iPad, or a `.zip` for an iOS simulator. You do not have to upload anything at all if your app is already published - you can install it on the device straight from Google Play, the App Store or TestFlight. ### Can I test an app that talks to my staging server? Yes. Start [TestingBot Tunnel](https://testingbot.com/support/tunnel) on your machine or a build agent and the device in our datacenter reaches your local or internal environment over an encrypted connection, without you exposing anything publicly. ### More reasons to love TestingBot ### 24/7 support Got questions? Ask in the 24/7 in-app customer [chat support](https://testingbot.com#) or [send us an email](https://testingbot.com/contact/new). We are here to help you! TestingBot Support Hey 👋 How can we help? I need support ### Trusted by over 14,000 teams 6M+ Tests 120k+ Users 400+ Enterprises 106 Countries [Request a demo](https://testingbot.com/demo) ### Developer Documentation Everything you need to integrate TestingBot into your workflow. Browse guides, API references and SDKs for web and mobile testing. - [Live Mobile App Testing documentation](https://testingbot.com/support/app-live) - [Upload your mobile app](https://testingbot.com/support/app-automate/help/upload) - [Preparing your app for testing](https://testingbot.com/support/app-automate/help/prepare) - [List of mobile devices](https://testingbot.com/support/app-automate/devices) [Explore Documentation](https://testingbot.com/support) ### Enterprise-grade Security STAR Level 1 Certified GDPR Compliant SAML SSO Single Sign-On 99.9% Uptime SLA [Visit TestingBot Trust Center](https://trust.testingbot.com/) ### Sign up for a Free Trial Start testing your apps with TestingBot. [Start a free trial](https://testingbot.com/users/sign_up) --- URL: https://testingbot.com/features/ai-testing # AI Test Automation Create automated tests in a fast and easy way. Generate tests in natural language with Gen AI. Generate your tests, replay on 6100+ browsers and mobile devices. [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) ![AI Test Automation](https://testingbot.com/assets/hero/codeless-hero-7afd44c329d9058be520bf755966ce28a6c6d7faef0bda267975991a751f8143.webp) Trusted by some of the world's most innovative companies ## AI Testing Describe in natural language what your test should do and let Gen AI create the test for you. Save time and resources from your developers and QA engineers by using AI to run tests against your website. The AI Testing Agent will convert your prompt into an automated test, which you can then schedule to run periodically on remote browsers. [Start free](https://testingbot.com/users/sign_up) ![AI Testing](https://testingbot.com/assets/features/ai-example-44d4424ca1c77ba88be8e28a761c4e0a8f1d056761a594cbed34ed2366f0e305.jpg) ## Example AI usecases to Test Your Website Use natural language prompts to check critical elements and flows on your site, using Generative AI. ### Shopping Cart Prompt: Add a product to the shopping cart. Verify on the checkout page that the product is in the shopping cart. ### Handle OCR/Captcha's Use the AI to handle OCR and captcha's. The AI will perform the necessary actions and validations. ### Skip Locators The AI will not depend on locators such as XPath or CSS selectors. Simply define elements by their text, such as "Click the button that says 'Submit'". ### Conditional Actions The AI will be able to perform conditional actions, such as "If the login is successful, check if the user is redirected to the dashboard page". ### Verify video playback Check if a video is playing correctly on the page. The AI agent will verify the video playback and check for any errors. ### Graph Check Prompt: Check that the page contains an image of a graph showing positive growth. ### Logo Presence Prompt: Verify that the company's logo is visible in the page header. ### CTA Button Check Prompt: Confirm the presence of a “Get Started” button that links to the signup page. ### Form Validation Prompt: Ensure that submitting an empty form displays an error message. ## AI Chat Chat with the AI Test Agent and instruct it to perform actions, such as clicking buttons, filling out forms, and verifying elements on the page. Ask the AI to generate automated test scripts from the actions you specified. Export to Java, Kotlin, Python, C#, Selenium IDE, JavaScript, Ruby and more. ## Record tests Import your recorded tests from Selenium IDE. TestingBot will run the tests periodically on the remote browsers you specified. [Start free](https://testingbot.com/users/sign_up) ![AI Testing](https://testingbot.com/assets/features/test-recorder-f95a63c80abaef9ba574444e53decc70f2c6304a9c0464339cc53637706eeeba.webp) ## AI Test Automation Run tests without writing any code on the TestingBot platform. ### Browsers & OS Select one or more browsers running on a specific operating system. Specify browser versions to target specific test environments. ### Schedule Run tests periodically; every minute, hour, day or at a custom interval. ### Alert When a test fails, you will receive email notifications or text messages. ### Reports Receive daily reports on the progress of your tests. ### Settings Set specific settings such as geolocation, screen resolution and more. ### Performance The tests will run in parallel on TestingBot's performant browser grid. ## Schedule Tests Schedule your tests to run on specific days and intervals. Set a cron format to define the exact days and hours you want your tests to run in parallel on our browsers. ![Schedule Tests](https://testingbot.com/assets/features/img-schedule-tests-5d62539793f48d4facd065f223691504c0aa9c36d3d394c2f2fa15969594f2e4.webp) ## Frequently Asked Questions ### How can I use AI to create automated tests? TestingBot allows you to use LLMs to create automated tests in natural language. Simply describe what you want to test, and the AI will generate the test for you. The AI will generate the test and instruct a remote browser to perform the necessary actions and validations. ### How can I record automated tests? You can use the official Selenium IDE browser plugin to record your test flow. The recorded test can be imported into TestingBot for execution. Once imported, you can choose the browsers and devices you want to run the test on, and schedule it to run periodically. ### What are some example AI prompts I can use to create tests? You can use prompts like "Create a test that logs into my website" or "Create a test that checks if the homepage loads correctly". Or check if an item can be added to a shopping cart - "Create a test that adds an item to the shopping cart and checks if the cart is updated correctly". ### Why should I use AI to create automated tests? You can write tests in plain English. Some customers even import their user stories from other platforms such as TestRail, PractiTest or Zephyr into TestingBot AI. It is very easy to explain and understand what is being tested. Because the TestingBot AI tests are not dependent on XPath or other brittle locators, they are ultra-stable. Customers are running hundreds of tests multiple times a day without a single failure. ### Ready to start AI Testing? Check out the [codeless testing documentation](https://testingbot.com/support/web-automate/codeless-automation/add-test) to get started. [View documentation](https://testingbot.com/support/web-automate/codeless-automation/add-test) Resources you may like [![Generative AI test agent](https://testingbot.com/assets/blog/2025/testingbot-ai.webp)](https://testingbot.com/blog/testingbot-ai-agent) ### [Generative AI test agent](https://testingbot.com/blog/testingbot-ai-agent) TestingBot has launched an AI Test Agent that will help you to create automated tests [Read more: Generative AI test agent](https://testingbot.com/blog/testingbot-ai-agent) [![Codeless Web Automation on iOS and Android](https://testingbot.com/assets/blog/2020/seleniumide.webp)](https://testingbot.com/blog/codeless-automation-ios-android) ### [Codeless Web Automation on iOS and Android](https://testingbot.com/blog/codeless-automation-ios-android) We now offer Codeless Automation for websites on iOS Simulators and Android Emulators. [Read more: Codeless Web Automation on iOS and Android](https://testingbot.com/blog/codeless-automation-ios-android) ### More reasons to love TestingBot ### 24/7 support Got questions? Ask in the 24/7 in-app customer [chat support](https://testingbot.com#) or [send us an email](https://testingbot.com/contact/new). We are here to help you! TestingBot Support Hey 👋 How can we help? I need support ### Trusted by over 14,000 teams 6M+ Tests 120k+ Users 400+ Enterprises 106 Countries [Request a demo](https://testingbot.com/demo) ### Developer Documentation Everything you need to integrate TestingBot into your workflow. Browse guides, API references and SDKs for web and mobile testing. - [Quickstart tutorials for Selenium](https://testingbot.com/support/web-automate/selenium) - [Appium Examples](https://testingbot.com/support/app-automate/appium) - [Integrations](https://testingbot.com/support/integrations) - [List of browsers & devices](https://testingbot.com/support/web-automate/browsers) [Explore Documentation](https://testingbot.com/support) ### Enterprise-grade Security STAR Level 1 Certified GDPR Compliant SAML SSO Single Sign-On 99.9% Uptime SLA [Visit TestingBot Trust Center](https://trust.testingbot.com/) --- URL: https://testingbot.com/features/accessibility-testing # Web Accessibility Testing TestingBot provides automated accessibility testing to help ensure your web applications comply with WCAG standards and are accessible to all users. Schedule recurring tests, receive detailed reports with actionable insights, and improve your application's accessibility with ease. [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) ![Web Accessibility Testing](https://testingbot.com/assets/hero/accessibility_testing-16c85de4dea69c5f2f0c0e421aa1f397623dbc7ec1a7ef3066965e8ad711736e.webp) Trusted by some of the world's most innovative companies ## Automated Accessibility Testing Identify and fix accessibility issues across your websites and web apps. Ensure WCAG compliance and deliver an inclusive experience for all users. ### WCAG Compliance Test your web applications against Web Content Accessibility Guidelines (WCAG) to ensure compliance. ### Scheduled Testing Configure one-time or recurring tests on a daily, weekly, or monthly basis to catch accessibility issues early. ### Detailed Reports Receive comprehensive reports with detailed information about each accessibility violation and recommended fixes. ### Customizable Alerts Set up notifications via email, webhooks, or SMS when accessibility issues are detected. ### Severity Classification Prioritize accessibility issues with clear severity labels to focus on the most critical problems first. ### Secure Testing Test locally hosted and authenticated pages securely using the [TestingBot Tunnel](https://testingbot.com/support/tunnel). ## Accessibility Testing is Essential Make your web applications accessible to all users and comply with regulations. ### Legal Compliance Meet legal requirements, such as the European Accessibility Act (EAA) to avoid any penalties. ### Broader Audience Reach more users by making your applications accessible to people with disabilities, who represent about 15% of the global population. ### Improved SEO Boost your search engine rankings as accessibility improvements often align with SEO best practices. ### Better User Experience Create a better experience for all users by implementing accessibility best practices that often improve usability for everyone. ## Detailed Accessibility Reports Receive comprehensive reports that identify accessibility issues with detailed information about each violation, its severity and recommendations for fixing the issue. [Start free](https://testingbot.com/users/sign_up) ![Detailed Accessibility Reports](https://testingbot.com/assets/features/accessibility/accessibility_ui-b64827a2253e7d403626e287a9da43bbf597d15b8e61fce56a53d8f3ffb9db1a.jpg) ![WCAG Compliance Testing](https://testingbot.com/assets/features/accessibility/wcag-45f2df574fac1c63f5ef8d3c7d437ba263d4feeff1be18bb950191d751228969.jpg) ## WCAG Compliance Testing Automatically test your web applications against WCAG guidelines to ensure they are accessible to users with disabilities and comply with international standards. [Read more](https://testingbot.com/support/accessibility/web/rules) ## Scheduled Accessibility Tests Configure one-time or recurring accessibility tests on a daily, weekly, or monthly basis to continuously monitor your web applications for accessibility issues. [Read more](https://testingbot.com/support/accessibility/web/scheduled) ![Scheduled Accessibility Tests](https://testingbot.com/assets/features/accessibility/accessibility_alert-aead51f69a0a303862fe064ebb28620f6505bbd2d26bf284c5c1727c82f4a719.jpg) ![WCAG Compliance Testing](https://testingbot.com/assets/features/accessibility/accessibility_code-bdee9af73b2c74be656f5067384af81c2302d810e6a314ed96d9978116e53e9f.jpg) ## Integrate into your existing tests With a few lines of code, you can check for accessibility issues during your automated Selenium tests. No custom SDK, library or framework required. [Read more](https://testingbot.com/support/accessibility/web/selenium) ### Sign up for a Free Trial Start testing your apps with TestingBot. [Start a free trial](https://testingbot.com/users/sign_up) Resources you may like [![Why is accessibility testing important in web design?](https://testingbot.com/assets/resources/articles/49-ddec8d6d134f657a98d9537849a0bb94fe908d49159be466aa83da15760120a4.webp)](https://testingbot.com/resources/articles/why-accessibility-testing) ### [Why is accessibility testing important in web design?](https://testingbot.com/resources/articles/why-accessibility-testing) Why accessible design matters, what the four POUR principles ask of you, and how to check a site against them as part of normal testing. [Read more: Why is accessibility testing important in web design?](https://testingbot.com/resources/articles/why-accessibility-testing) [![Automated Accessibility Testing with TestingBot](https://testingbot.com/assets/blog/2021/accessibility.webp)](https://testingbot.com/blog/automated-accessibility-testing) ### [Automated Accessibility Testing with TestingBot](https://testingbot.com/blog/automated-accessibility-testing) Start testing for accessibility issues with TestingBot and Axe. [Read more: Automated Accessibility Testing with TestingBot](https://testingbot.com/blog/automated-accessibility-testing) --- URL: https://testingbot.com/features/headless # Headless Browsers Connect AI agents and test scripts to remote, headless browsers. Perform automated browser actions, simulate user interactions or perform data-driven tasks with full browser capabilities and zero setup. [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) ![Headless Browsers](https://testingbot.com/assets/features/headless/headless-hero-38d22aa0dac3f4e755ca7d810be1172495f2adaa52c25f1a2efa423ca6b74ede.webp) Trusted by some of the world's most innovative companies ![Test on real mobile and desktop devices](https://testingbot.com/assets/features/headless/headless-browsers-39b7c3acbb683809c7f8ae34359638d62c04784315d3435def57f0911013278b.jpg) ## How it works TestingBot takes care of the setup and maintenance of the headless browsers. Run your sessions in parallel on our cloud-based grid, optimized for speed and performance. Take advantage of a high performance platform, with low latency and fast browser startup times. Extract HTML, bypass bot detectors and use user-in-the-loop functionality with Hybrid Automations. [Start free](https://testingbot.com/users/sign_up) ## Browser Frameworks These automation frameworks support running headless sessions. [![Selenium Testing](https://testingbot.com/assets/support/selenium-266ecc0599068fc6eca0bcd9174fb46053c95cebe1c66f7a50c480f3f1a52929.svg)](https://testingbot.com/features/automation/selenium) ### [Selenium](https://testingbot.com/features/automation/selenium) Popular Browser Automation Framework, using WebDriver, which supports all major browsers. [![Puppeteer Testing](https://testingbot.com/assets/support/puppeteer-0a8cdd38af69dabd46638afbb3bffc8a372902949aa599663ec379f3e875263c.svg)](https://testingbot.com/features/automation/puppeteer) ### [Puppeteer](https://testingbot.com/features/automation/puppeteer) Chrome DevTools based Automation Framework by Google. Supports Chrome and Edge browsers. [![Playwright Testing](https://testingbot.com/assets/support/playwright-2dcb4e75b55ac0d1135cfa47be6137b13f46a2047ce78b96a34f51b0e2cc0d96.svg)](https://testingbot.com/features/automation/playwright) ### [Playwright](https://testingbot.com/features/automation/playwright) Similar to Puppeteer, built by Microsoft, with support for Chrome and Edge. ![Instrument native, remote browsers with speed.](https://testingbot.com/assets/features/manual-flow-b2c093218b8db3363e5903a6197ce47efa4361cd8c114738489bbc673b5d168a.jpg) ## Instrument native, remote browsers with speed. Take advantage of increased performance and speed with headless browsers. Test, scrape or fetch information from websites with real browsers. Feed data to your AI agents. [Start free](https://testingbot.com/users/sign_up) ## Parallel Sessions Drastically improve your workflow by running sessions simultaneously. TestingBot provides single-use machines on Windows, Linux and macOS, optimized for speed and stability. [Start free](https://testingbot.com/users/sign_up) ![Cut suite time from hours to minutes](https://testingbot.com/assets/features/selenium/parallel-0229d698e59a8a6b92fe50344a08634624f018331ecf8a849ef83cfb35e635a5.png) ## Video, screenshots & logs Every browser comes with access to generated logs, monitoring, videos and screenshots. [Start free](https://testingbot.com/users/sign_up) ![Video, screenshots & logs](https://testingbot.com/assets/features/app-automated-report-dcef51604982dffa3ee9f02b87a92cef82132f9e21c5b66683738ab69f634ecf.jpg) ![WCAG Compliance Testing](https://testingbot.com/assets/features/headless/headless-scripts-dab9e6afb5d1bc095e14efd5f2054cfb36b50c5dea0be0ec20a9120f21097703.webp) ## Remote Browser Functions Use TestingBot Functions to easily scrape URLs, automate actions, generate PDFs/screenshots and more. [Read more](https://testingbot.com/support/functions) ### More reasons to love TestingBot ### 24/7 support Got questions? Ask in the 24/7 in-app customer [chat support](https://testingbot.com#) or [send us an email](https://testingbot.com/contact/new). We are here to help you! TestingBot Support Hey 👋 How can we help? I need support ### Trusted by over 14,000 teams 6M+ Tests 120k+ Users 400+ Enterprises 106 Countries [Request a demo](https://testingbot.com/demo) ### Developer Documentation Everything you need to integrate TestingBot into your workflow. Browse guides, API references and SDKs for web and mobile testing. - [Quickstart for Playwright](https://testingbot.com/support/web-automate/playwright) - [Run Puppeteer on TestingBot](https://testingbot.com/support/web-automate/puppeteer) - [Functions: scrape, take screenshots and serverless functions](https://testingbot.com/support/functions) - [Playwright Recorder](https://testingbot.com/support/web-automate/playwright/recorder) [Explore Documentation](https://testingbot.com/support) ### Enterprise-grade Security STAR Level 1 Certified GDPR Compliant SAML SSO Single Sign-On 99.9% Uptime SLA [Visit TestingBot Trust Center](https://trust.testingbot.com/) ### Ready to start your first headless browser session? Please see the [Playwright documentation](https://testingbot.com/support/web-automate/playwright) to get started. [View documentation](https://testingbot.com/support/web-automate/playwright) Resources you may like [![Automated Testing with Puppeteer](https://testingbot.com/assets/resources/articles/11-416015eb394d928299663409dccd67993b0a664f4b3ff456803fefc6bf1f325a.webp)](https://testingbot.com/resources/articles/automated-testing-with-puppeteer) ### [Automated Testing with Puppeteer](https://testingbot.com/resources/articles/automated-testing-with-puppeteer) Puppeteer combined with a test framework provides a great way to run automated browser tests. Follow this guide for more information. [Read more: Automated Testing with Puppeteer](https://testingbot.com/resources/articles/automated-testing-with-puppeteer) [![Visual Testing with Playwright](https://testingbot.com/assets/resources/articles/14-f3f2e8954c8a47cf0bb29059b44f4d2f40998442e22db6f8acdccb8e5b34d8c1.webp)](https://testingbot.com/resources/articles/playwright-visual-regression-testing) ### [Visual Testing with Playwright](https://testingbot.com/resources/articles/playwright-visual-regression-testing) Playwright provides automated browser testing. It offers a built-in feature to perform visual regression testing for your website. [Read more: Visual Testing with Playwright](https://testingbot.com/resources/articles/playwright-visual-regression-testing) [![Puppeteer Testing in the Cloud](https://testingbot.com/assets/blog/2021/puppeteer.webp)](https://testingbot.com/blog/puppeteer-cloud-testing) ### [Puppeteer Testing in the Cloud](https://testingbot.com/blog/puppeteer-cloud-testing) Start running Puppeteer tests in the cloud with TestingBot. [Read more: Puppeteer Testing in the Cloud](https://testingbot.com/blog/puppeteer-cloud-testing) --- URL: https://testingbot.com/features/parallel-testing # Parallel testing Parallel testing is running several automated tests at the same time instead of one after another. Each test gets its own isolated browser or device session, so a suite that takes an hour sequentially can finish in a few minutes. The number you can run at once is your concurrency. [See plans by concurrency](https://testingbot.com/pricing) ## How concurrency works on TestingBot Every test you start takes one parallel session for as long as it runs, then releases it. Your plan sets how many sessions can be open at once. Browser sessions and real-device sessions draw on the same pool, so a Selenium run and an Appium run compete for the same slots. If your CI job starts more tests than your plan allows, the extra ones queue instead of failing. TestingBot holds them and starts each as a slot frees up, so a burst above your concurrency still completes. It just takes longer, which is usually the first sign that it is time to size up. ### Parallel sessions by plan | Plan | Parallel sessions available | | --- | --- | | Automated | 1, 2, 4, 8, 16, 24 | | Pro | 1, 2, 4, 8, 16, 24 | | Live (manual) | 1, 2, 4, 10, 25 | Concurrency is the axis these plans scale on. For what each tier costs, see [pricing](https://testingbot.com/pricing). ## What speedup to actually expect A suite of 200 tests averaging one minute each takes about 200 minutes end to end. Split across parallel sessions it drops roughly in proportion, until the point where your slowest individual test becomes the floor. Nothing finishes faster than its longest single test, however many sessions you buy. | Parallel sessions | 200-minute suite finishes in about | | --- | --- | | 1 | 200 min | | 2 | 100 min | | 4 | 50 min | | 8 | 25 min | | 16 | 13 min | | 24 | 9 min | Indicative only. Real suites have uneven test durations, so a run ends when the slowest worker finishes rather than when the average one does. Treat these as an upper bound and expect to land slightly above each figure. To size it against your own suite rather than this example, use the [parallel test calculator](https://testingbot.com/support/parallel-calculator). ## Running your framework in parallel Parallelism is a test-runner setting, not a rewrite, as long as your tests are independent of each other. Each framework spells it differently: - [Playwright parallel testing](https://testingbot.com/support/web-automate/playwright/playwright-parallel-testing) — set workers in your Playwright config and shard across CI machines. - [Selenium parallel testing with JUnit](https://testingbot.com/support/web-automate/selenium/java/parallel-junit) — thread-safe WebDriver instances and parallel JUnit execution. - [Puppeteer parallel testing](https://testingbot.com/support/web-automate/puppeteer/puppeteer-parallel-testing) — run multiple browser connections concurrently against the grid. - [Parallel test calculator](https://testingbot.com/support/parallel-calculator) — work out how many sessions your suite needs to hit a target run time. Framework overviews: [Selenium](https://testingbot.com/features/automation/selenium), [Playwright](https://testingbot.com/features/automation/playwright), [Cypress](https://testingbot.com/features/automation/cypress) and [Appium](https://testingbot.com/features/automation/appium). ## Frequently asked questions - **What is parallel testing?:** Parallel testing is running several automated tests at the same time instead of one after another. Each test gets its own isolated browser or device session, so a suite that takes an hour sequentially can finish in a fraction of that. The number of tests you can run at once is called your concurrency, or your number of parallels. - **How many parallel tests do I need?:** Divide your total suite runtime by the wall-clock time you are willing to wait, then round up. A 200-minute suite that must finish inside 20 minutes needs roughly 10 parallels. Adding parallels past that point stops helping, because your slowest single test becomes the floor. - **Does doubling my parallels halve my test run?:** Close to it at first, but not exactly, and not forever. Real suites have uneven test durations, so the run ends when the slowest worker finishes rather than when the average one does. Speedup is close to linear while you have many more tests than parallels, and flattens as the two numbers converge. - **What happens if I start more tests than my plan allows?:** They queue rather than fail. TestingBot holds the extra sessions until a slot frees up, then starts them automatically, so a CI job that bursts above your concurrency still completes without errors. It simply takes longer. - **Do I need to change my test code to run in parallel?:** You change your test runner configuration, not your tests, as long as your tests are independent. Each framework has its own switch: workers in Playwright, threads in TestNG or pytest-xdist, and the parallel flag in Cypress. Tests that share state or fixed test data need isolating first. - **Does parallel testing work with real mobile devices?:** Yes. Appium, XCUITest, Espresso and Maestro sessions consume the same concurrency as browser sessions, so the same plan covers both. Each parallel session gets its own dedicated device. --- URL: https://testingbot.com/features/screenshot-testing # Automated Visual UI Regression Testing Detect and fix visual issues on your webpages and mobile apps before they impact users. Leverage TestingBot's codeless visual UI testing to get instant alerts whenever a visual discrepancy is detected on your site. [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) ![Automated Visual UI Regression Testing](https://testingbot.com/assets/hero/visual-abbead5878a5c7c5d80577279ba1f4c14a31df4ffdb0309dad7f6b7c3b09e63e.svg) Trusted by some of the world's most innovative companies ![Test on real mobile and desktop devices](https://testingbot.com/assets/features/screenshot-command-8a1297c081a302dc4dbc77a4e4461dec802ee7f7cfc8153948e2e06ee30b891e.jpg) ## Effortless Visual Regression Testing Enable visual regression testing in your existing test framework with a single line of code. Quickly detect layout shifts, UI inconsistencies and visual regressions during your test. Or enter the URLs you would like to monitor. TestingBot will take screenshots on regular intervals and alert you when a change is detected. [Start free](https://testingbot.com/users/sign_up) ## Review and Approve Visual Regressions You and your team can review visual test results, approving or rejecting specific UI changes. Use a slider to compare the baseline and new screenshots or analyze a diff image highlighting the differences. ![Test on real mobile and desktop devices](https://testingbot.com/assets/features/visual_schedule-8bc03556f3225d26b8fd4e5bdb31c1881ef2abb3257c35093cc441b5b62abbaa.jpg) ## Codeless Visual UI Testing Start identifying UI defects on your website in minutes. Just enter your URL, select the browsers you want to test on, and set the testing frequency. Customize alerts to get notified instantly when issues are detected. When a visual change occurs, you'll receive an instant alert with a screenshot and a diff image highlighting the differences. Keep your UI and UX flawless by catching unexpected changes early. [Start free](https://testingbot.com/users/sign_up) ## Optimized for accuracy We built visual testing with accuracy in mind, preventing false-positives during the UI verification process. ### Freeze animations and videos Animations and videos are temporarily frozen during the visual snapshot, to prevent false positives. ### Ignore UI elements We ignore certain UI elements that are prone to change and are irrelevant to the screenshot. ### Pixel Perfect Screenshots Take full-page, above the fold or chromeless screenshots of your website across hundreds of browsers and devices at once. ## Test Automation Frameworks TestingBot's UI testing seamlessly integrates with Selenium, Appium, Puppeteer and Playwright. Just add a single line of code to enhance your existing tests with automated visual regression testing. [![Selenium Testing](https://testingbot.com/assets/support/selenium-266ecc0599068fc6eca0bcd9174fb46053c95cebe1c66f7a50c480f3f1a52929.svg)](https://testingbot.com/support/visual-testing/automated/selenium) ### [Selenium](https://testingbot.com/support/visual-testing/automated/selenium) Popular Browser Automation Framework, using WebDriver, which supports all major browsers. [![Appium Testing](https://testingbot.com/assets/support/appium-4b35a8bb56bae236bd84c539cee9b8bbb489a55ddf48e32d059c86685d230354.svg)](https://testingbot.com/support/visual-testing/automated/appium) ### [Appium](https://testingbot.com/support/visual-testing/automated/appium) Mobile Automation Framework, using WebDriver, runs on physical devices and simulators/emulators. [![Playwright Testing](https://testingbot.com/assets/support/playwright-2dcb4e75b55ac0d1135cfa47be6137b13f46a2047ce78b96a34f51b0e2cc0d96.svg)](https://testingbot.com/support/visual-testing/automated/playwright) ### [Playwright](https://testingbot.com/support/visual-testing/automated/playwright) Similar to Puppeteer, built by Microsoft, with support for Chrome and Edge. [![Puppeteer Testing](https://testingbot.com/assets/support/puppeteer-0a8cdd38af69dabd46638afbb3bffc8a372902949aa599663ec379f3e875263c.svg)](https://testingbot.com/support/visual-testing/automated/puppeteer) ### [Puppeteer](https://testingbot.com/support/visual-testing/automated/puppeteer) Chrome DevTools based Automation Framework by Google. Supports Chrome and Edge browsers. ## Frequently Asked Questions ### How can I use this to spot visual defects? TestingBot performs automated pixel-by-pixel comparisons between an image that you marked as correct and a new screenshot that was just taken. If the amount of different pixels exceeds the threshold, the test is considered as failed. This can happen when you deploy a new version of your website that has an incorrect CSS styling, positioning or is missing from the page. Catch visual bugs before these cause harm to the UX of your webpages. ### Why can’t functional tests cover visual problems? You can test for dimensions, positioning and styling with regular functional tests. The disadvantage is that it will soon become a huge burden of checkpoints in your code. Instead of doing the visual checks in your code, it is much more efficient to take snapshots and compare these. The visual results are also much more easily interpreted by developers and QA. ### When Should I Use Visual Testing? Ideally you would run visual checks as part of your CI/CD pipeline. It could be run after each check-in of front-end code, making sure new code does not negatively impact your product. ### Can AI be used for visual testing? Machine learning algorithms can be trained to discover visual differences. AI depends on trained data, so it may report false positives/negatives. It is not a Not a One-Size-Fits-All solution and often fails in comparison with exact pixel matching. ### How does Snapshot Testing work? Snapshot Testing in the context of visual testing for UI on webpages, captures screenshots of the initial state of a webpage. It then compares them against subsequent renders. Tests are considered failed when one or more changes in visual appearance occur. This helps to ensure that UI elements remain consistent. ### Sign up for a Free Trial Start visually testing your apps and websites with TestingBot. [Start a free trial](https://testingbot.com/users/sign_up) Resources you may like [![Visual Regression Testing with Python](https://testingbot.com/assets/resources/articles/20-823a333072d87c93f6d75a502e3073e32bd0bee1d3142540ce0b42da6fa7ccf0.webp)](https://testingbot.com/resources/articles/python-visual-testing) ### [Visual Regression Testing with Python](https://testingbot.com/resources/articles/python-visual-testing) How to add visual regression testing to a Python suite: creating a baseline, comparing against it, and keeping screenshots stable over time. [Read more: Visual Regression Testing with Python](https://testingbot.com/resources/articles/python-visual-testing) [![Angular UI Testing](https://testingbot.com/assets/resources/articles/21-15af461a46c449ef26dfb952ed9d0fc327c97d9ca815f24f8fe9ee60b9aa64a5.webp)](https://testingbot.com/resources/articles/angular-ui-testing) ### [Angular UI Testing](https://testingbot.com/resources/articles/angular-ui-testing) How to add visual regression testing to an Angular app with Cypress, comparing screenshots against a baseline to catch unintended UI changes. [Read more: Angular UI Testing](https://testingbot.com/resources/articles/angular-ui-testing) [![Visual Regression Testing](https://testingbot.com/assets/blog/2024/visual-testing.webp)](https://testingbot.com/blog/visual-regression-testing) ### [Visual Regression Testing](https://testingbot.com/blog/visual-regression-testing) Learn about TestingBot's visual testing feature, allowing you to easily compare layouts across browsers and devices. [Read more: Visual Regression Testing](https://testingbot.com/blog/visual-regression-testing) --- URL: https://testingbot.com/features/automation/puppeteer Puppeteer cloud # Run Puppeteer on a real browser cloud Point your existing Puppeteer scripts at TestingBot's grid and run them across real Chrome and Edge on Windows, macOS and Linux. Massive parallelism, video & log artifacts, GDPR-compliant with EU data residency. Choosing a framework? See [Selenium vs Puppeteer](https://testingbot.com/selenium-vs-puppeteer) for a head-to-head breakdown, or [Playwright vs Selenium](https://testingbot.com/playwright-vs-selenium). [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) - **Browsers & OSes:** Chrome · Edge - **Parallel sessions:** 100 - **Uptime SLA:** 99.9% Trusted by some of the world's most innovative companies 30-second answer ## Why run Puppeteer on TestingBot? Real Chrome and Edge TestingBot runs your Puppeteer scripts against real Chrome and Edge on real Windows, macOS and Linux machines. No more "works in headless Chromium on my laptop" surprises. Massive parallelism Fan out to up to 100 concurrent Puppeteer sessions, each in its own single-use VM. Headless or headful, your choice per session. Recorder & extras Use the Puppeteer Recorder to generate scripts without writing a line of code. Test browser extensions, web scrape and run cross-browser parallel suites from the same dashboard. What is Puppeteer? ## Puppeteer in 60 seconds ![Puppeteer](https://testingbot.com/assets/support/puppeteer-0a8cdd38af69dabd46638afbb3bffc8a372902949aa599663ec379f3e875263c.svg) ### Puppeteer Released 2017 · Chrome DevTools team · Apache 2.0 Puppeteer is a Node.js library from the Chrome DevTools team that drives Chrome and Chromium through the DevTools Protocol. It was the original "headless Chrome" automation library and remains the fastest way to script Chrome from JavaScript. Puppeteer ships first-class APIs for screenshots, PDFs, network interception, performance traces and page metrics. It is the canonical choice for Chrome-first end-to-end testing, web scraping and headless rendering pipelines. - Node.js library, JavaScript / TypeScript - Chrome, Chromium and Edge via the DevTools Protocol - First-class screenshots, PDFs and network interception - Headless and headful, single API - Maintained by the Chrome DevTools team 5-line setup ## Point your scripts at the TestingBot grid Reuse your existing Puppeteer code. Connect to TestingBot's WebSocket endpoint and your scripts run on real Chrome and Edge in parallel. connect.js step 1 ```javascript // Connect Puppeteer to TestingBot's cloud const puppeteer = require('puppeteer-core'); const caps = { browserName: 'chrome', browserVersion: 'latest', platform: 'WIN11', 'tb:options': { key: 'KEY', secret: 'SECRET' }, }; const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com?caps=${ encodeURIComponent(JSON.stringify(caps))}`, }); ``` login.spec.js step 2 ```javascript // Your existing Puppeteer script, unchanged const page = await browser.newPage(); await page.goto('https://app.example.com/login'); await page.type('#user', 'jane'); await page.type('#pwd', '••••'); await Promise.all([ page.click('#go'), page.waitForNavigation(), ]); await page.screenshot({ path: 'dashboard.png' }); await browser.close(); ``` See the full [Puppeteer on TestingBot documentation](https://testingbot.com/support/web-automate/puppeteer) for capabilities, video recording and parallel options. Local vs cloud ## Local Puppeteer vs TestingBot Puppeteer is fast on a laptop until you need scale, multiple OSes or real Edge. Here is how a hosted cloud compares. | Capability | TestingBot | Local Puppeteer | | --- | --- | --- | | Browser coverage | Real Chrome and Edge across versions | Whatever you have installed locally | | Operating systems | Real Windows, macOS and Linux | Your single dev machine | | Parallel scaling | Up to 100 concurrent sessions, on demand | Capped by your CPU and RAM | | Video & logs | HD video, console logs, screenshots included | Roll your own recording & log capture | | Test private & staging environments | Built-in secure tunnel | Direct access only, no secure remote sharing | | EU data residency | Yes, GDPR-compliant | Whatever country your laptop is in | | Reliability | 99.9% uptime SLA, 24/7 monitoring | Crashes when you close the lid | ![Video, screenshots and logs out of the box](https://testingbot.com/assets/features/test-results-315382c39a67ad52167d2a4fd78570c99aea3e0ec386c17e82265851e93cb7b3.png) Test artifacts ## Video, screenshots and logs out of the box Every Puppeteer session ships with a full HD video, browser console log and screenshots. Inspect a failure without re-running the test. Mark sessions as passed or failed from your script, fetch artifacts via our [REST API](https://testingbot.com/support/api), and ship results back to Jira, Slack or your CI of choice. - Full HD video for every session - Console and DevTools logs - Screenshots and PDFs on demand - Custom screen resolution, timezone and locale - Browser startup flags and [many more options](https://testingbot.com/support/web-automate/puppeteer/options) Languages & runners ## Puppeteer integrations for every stack Puppeteer is Node-first, but the same scripts run from Jest, WebDriverIO and CodeceptJS, plus Python via Pyppeteer and Go via chromedp. All point at the same TestingBot endpoint. - ### [Node.js](https://testingbot.com/support/web-automate/puppeteer/capabilities) - [Quickstart](https://testingbot.com/support/web-automate/puppeteer) - [Capabilities](https://testingbot.com/support/web-automate/puppeteer/capabilities) - [Browser flags](https://testingbot.com/support/web-automate/puppeteer/options) - [Test results](https://testingbot.com/support/web-automate/puppeteer/test-results) - ### [Jest](https://testingbot.com/support/web-automate/puppeteer/jest) - [Jest with Puppeteer](https://testingbot.com/support/web-automate/puppeteer/jest) - ### [WebDriverIO](https://testingbot.com/support/web-automate/puppeteer/webdriverio) - [WebDriverIO + Puppeteer](https://testingbot.com/support/web-automate/puppeteer/webdriverio) - ### [CodeceptJS](https://testingbot.com/support/web-automate/puppeteer/codeceptjs) - [CodeceptJS setup](https://testingbot.com/support/web-automate/puppeteer/codeceptjs) - ### [Python · PyTest](https://testingbot.com/support/web-automate/puppeteer/pytest) - [PyTest with Pyppeteer](https://testingbot.com/support/web-automate/puppeteer/pytest) - ### [Go · chromedp](https://testingbot.com/support/web-automate/puppeteer/chromedp) - [chromedp setup](https://testingbot.com/support/web-automate/puppeteer/chromedp) Looking for use-case guides? Read our [Puppeteer Recorder](https://testingbot.com/support/web-automate/puppeteer/recorder), [web scraping](https://testingbot.com/support/web-automate/puppeteer/scraping), [browser extension testing](https://testingbot.com/support/web-automate/puppeteer/browser-extension-testing) and [parallel testing](https://testingbot.com/support/web-automate/puppeteer/puppeteer-parallel-testing) guides. Massive parallelism ## Cut Puppeteer suite time from hours to minutes Run hundreds of Puppeteer sessions in parallel across single-use VMs. Headless or headful, same scripts, a fraction of the wall-clock time. - Up to 100 concurrent Puppeteer sessions - Sessions start in seconds, no warm-up - Headless or headful, switchable per session [Read more](https://testingbot.com/support/web-automate/puppeteer/puppeteer-parallel-testing) ![Cut Puppeteer suite time from hours to minutes](https://testingbot.com/assets/features/selenium/parallel-0229d698e59a8a6b92fe50344a08634624f018331ecf8a849ef83cfb35e635a5.png) Live exit nodes 10+ countries United States United Kingdom Germany France Spain Belgium Australia China Japan Geolocation testing ## Test Puppeteer scripts from 10+ countries Run Puppeteer scripts from real exit nodes around the world. Verify **geo-targeting** , **geo-blocking** and **localisation** from the United States, the United Kingdom, Germany, France, Spain, Belgium, Australia, China, Japan and more. Secure infrastructure ## Real machines. Single-use. Zero leaks. Each Puppeteer session runs in a brand-new, single-use virtual machine on a real operating system. The VM is destroyed the moment your script ends: nothing is reused, nothing leaks across customers. GDPR-compliant with EU data residency, SAML SSO and audit logs available for enterprise teams. - Single-use VMs, wiped after every session - GDPR-compliant, EU data residency available - SAML SSO, audit logs and SCIM provisioning [Visit TestingBot Trust Center](https://trust.testingbot.com/) 1. 1 Script connects to `cloud.testingbot.com` A pristine VM is provisioned in seconds, just for your session. 2. 2 Puppeteer drives a real Chrome or Edge Real Windows, macOS or Linux, never simulated, never shared. 3. 3 VM destroyed the moment your script ends Cookies, files, browser state, all wiped. Nothing is ever reused. Continuous integration ## Plug Puppeteer into your CI/CD in minutes TestingBot integrates with every major CI/CD tool. Trigger Puppeteer runs on every commit and stream videos and logs back into your build dashboards. ![Jenkins](https://testingbot.com/assets/integrations/jenkins-e586a1c4af833e4c8d38d7b392a4e84bd193f30cae9cf8c9f198a58ec943a1ff.webp) ### [Jenkins](https://testingbot.com/support/integrations/ci-cd/jenkins) ![Bamboo](https://testingbot.com/assets/integrations/bamboo-c8fe09a3cb0b74d6055e3aa6607d77720bfe29ff487b8e3cf825138adf634346.svg) ### [Bamboo](https://testingbot.com/support/integrations/ci-cd/bamboo) ![TeamCity](https://testingbot.com/assets/integrations/teamcity-7e75cdb835903a4983ad276c41d95e569ceac5be486e96ec1f955029ce5bb319.svg) ### [TeamCity](https://testingbot.com/support/integrations/ci-cd/teamcity) ![CircleCI](https://testingbot.com/assets/integrations/circleci-ede029654ff39b60342e4a5bba8fc9a3a49305cc38bda215b7cf8d07e85e0543.webp) ### [CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci) ![GitLab CI](https://testingbot.com/assets/integrations/gitlab-e4c0b6db07d07bc0a400f5606e258f4dc8a51b778e97bbfa51d1d531f3cb049a.webp) ### [GitLab CI](https://testingbot.com/support/integrations/ci-cd/gitlab) ![GitHub](https://testingbot.com/assets/integrations/github-actions-8436242ad9e15381f84f1a98a10dddce15611087f121d6305782e211942b7842.webp) ### [GitHub](https://testingbot.com/support/integrations/ci-cd/github-actions) Secure Tunnel ## Test internal & staging apps securely Use our [secure tunnel](https://testingbot.com/support/tunnel) to run Puppeteer scripts against firewalled staging sites, internal dashboards or apps still on localhost. Encrypted end-to-end, no inbound firewall rules required, ready out of the box for popular CI providers. ![TestingBot secure tunnel for Puppeteer](https://testingbot.com/assets/features/tunnel-diagram-poster-8c25e19e6fe4f7276a46a96b31a0ad0e07bb4f5bf98dc192d2b70897cfbb1a26.png) [Read more](https://testingbot.com/support/tunnel) FAQ ## Puppeteer on TestingBot — frequently asked questions Why run Puppeteer on TestingBot instead of locally? Locally you are capped by your laptop or CI runner. TestingBot gives you up to 100 parallel Puppeteer sessions on real Chrome and Edge across real Windows, macOS and Linux machines, with HD video and console logs collected automatically. Which Puppeteer versions are supported? TestingBot supports every supported Puppeteer release. You can pin a specific Puppeteer version per-test using capabilities. Can I run Puppeteer tests in parallel? Yes. Plans on TestingBot include up to 100 parallel Puppeteer sessions. Each session runs in its own single-use VM, so scripts do not interfere with one another. Can I use Puppeteer with Jest, WebDriverIO or CodeceptJS? Yes. We have dedicated guides for [Jest](https://testingbot.com/support/web-automate/puppeteer/jest), [WebDriverIO](https://testingbot.com/support/web-automate/puppeteer/webdriverio) and [CodeceptJS](https://testingbot.com/support/web-automate/puppeteer/codeceptjs), plus Python via [PyTest](https://testingbot.com/support/web-automate/puppeteer/pytest) and Go via [chromedp](https://testingbot.com/support/web-automate/puppeteer/chromedp). How is Puppeteer different from Selenium and Playwright? Puppeteer is Chrome-first and Node-first, with a smaller, snappier API than Selenium and a narrower scope than Playwright. Pick Puppeteer when you only need Chrome/Edge and want speed, screenshots, PDFs and network control with minimum ceremony. TestingBot supports all three on the same grid. Can I record Puppeteer scripts without writing code? Yes, use the [Puppeteer Recorder](https://testingbot.com/support/web-automate/puppeteer/recorder) to capture clicks, types and assertions, then run the resulting script on the TestingBot grid. Can I test internal or staged sites? Yes, install our [secure tunnel](https://testingbot.com/support/tunnel) and your Puppeteer scripts can reach staging environments, localhost or anything else behind your firewall. Is the Puppeteer cloud GDPR / SOC 2 compliant? TestingBot is not SOC 2 or ISO 27001 certified. We are GDPR-compliant with EU-only data residency, registered at CSA STAR Level 1 (a published self-assessment), and our control set is modelled on ISO 27001 Annex A. Our datacenter operators hold ISO 27001. We offer SAML SSO, SCIM and audit logs on enterprise plans. ## Ready to scale your Puppeteer testing? Read our [Puppeteer Cloud Testing](https://testingbot.com/support/web-automate/puppeteer) documentation and run your first script in under 5 minutes. [Start a free trial](https://testingbot.com/users/sign_up) Resources you may like [![Automated Testing with Puppeteer](https://testingbot.com/assets/resources/articles/11-416015eb394d928299663409dccd67993b0a664f4b3ff456803fefc6bf1f325a.webp)](https://testingbot.com/resources/articles/automated-testing-with-puppeteer) ### [Automated Testing with Puppeteer](https://testingbot.com/resources/articles/automated-testing-with-puppeteer) Puppeteer combined with a test framework provides a great way to run automated browser tests. Follow this guide for more information. [Read more: Automated Testing with Puppeteer](https://testingbot.com/resources/articles/automated-testing-with-puppeteer) [![Puppeteer Testing in the Cloud](https://testingbot.com/assets/blog/2021/puppeteer.webp)](https://testingbot.com/blog/puppeteer-cloud-testing) ### [Puppeteer Testing in the Cloud](https://testingbot.com/blog/puppeteer-cloud-testing) Start running Puppeteer tests in the cloud with TestingBot. [Read more: Puppeteer Testing in the Cloud](https://testingbot.com/blog/puppeteer-cloud-testing) --- URL: https://testingbot.com/features/automation/playwright Playwright cloud # Run Playwright testing on a real browser cloud Point your existing Playwright suite at TestingBot's grid for cross-browser Playwright testing across 6100+ real browsers and Android devices. Massive parallelism, test artifacts, GDPR-compliant with EU data residency. Choosing a framework? See [Playwright vs Selenium](https://testingbot.com/playwright-vs-selenium) or [Cypress vs Playwright](https://testingbot.com/cypress-vs-playwright). [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) - **Browsers & devices:** 6100+ - **Parallel sessions:** 20+ - **Uptime SLA:** 99.9% Trusted by some of the world's most innovative companies 30-second answer ## Why run Playwright on TestingBot? Real browsers, not just Chromium TestingBot runs your Playwright suite against real Chrome, Firefox, Edge and WebKit on real Windows, macOS and Linux machines. Microsoft Playwright Testing caps you at Chromium on Linux. Predictable pricing Plans include unlimited test minutes. No per-minute meter, no separate reporting fee, no surprise bills the morning after a flaky CI run. Real Android devices Move past Playwright's mobile emulation and run your flows on real Android phones in our cloud, alongside the same desktop suite. What is Playwright? ## Playwright in 60 seconds ![Playwright](https://testingbot.com/assets/support/playwright-2dcb4e75b55ac0d1135cfa47be6137b13f46a2047ce78b96a34f51b0e2cc0d96.svg) ### Playwright Released 2020 · Microsoft · Apache 2.0 Playwright is an end-to-end testing framework built by Microsoft that drives Chromium, Firefox and WebKit through one API. It was created by the team that originally built Puppeteer, with cross-browser support and modern web app needs in mind. Auto-waiting, network interception, multi-tab support and browser contexts come out of the box. The official @playwright/test runner ships with the framework, so you do not need to bolt on Mocha, Jest or PyTest unless you want to. Teams use Playwright for end-to-end testing, cross-browser testing, visual regression testing, API testing, accessibility testing and component testing across a single codebase. - TypeScript / JavaScript / Python / Java / .NET bindings - Auto-wait, network interception, fixtures, traces - Chrome, Firefox, WebKit · headed and headless - Built-in test runner, parallel workers, retries - Video, screenshots and logs 5-line setup ## Point your tests at the TestingBot grid Reuse your existing Playwright code. Set the wsEndpoint to TestingBot's cloud and your tests run on real browsers, real OSes, in parallel. playwright.config.ts step 1 ```typescript // Connect to TestingBot's Playwright cloud import { defineConfig } from '@playwright/test'; export default defineConfig({ use: { connectOptions: { wsEndpoint: 'wss://cloud.testingbot.com/playwright? key=KEY&secret=SECRET&browser=chrome', }, }, workers: 10, trace: 'on', }); ``` login.spec.ts step 2 ```typescript // Your existing Playwright test, unchanged import { test, expect } from '@playwright/test'; test('login redirects to dashboard', async ({ page }) => { await page.goto('https://app.example.com/login'); await page.getByLabel('Username').fill('jane'); await page.getByLabel('Password').fill('••••'); await page.getByRole('button').click(); await expect(page).toHaveURL(/\/dashboard/); }); ``` See the full [Playwright on TestingBot documentation](https://testingbot.com/support/web-automate/playwright) for capabilities, video recording and options. TestingBot vs MPT ## TestingBot vs Microsoft Playwright Testing Microsoft Playwright Testing (MPT) is a useful service if you live entirely in the Microsoft stack. For everything else, TestingBot is the more complete cloud. | Capability | TestingBot | Microsoft Playwright Testing | | --- | --- | --- | | Browsers | Real Chrome, Firefox, Edge, WebKit on real OSes | Chromium on Linux only | | Operating systems | Real Windows, macOS and Linux | Windows and Linux only | | Pricing | Flat plans with unlimited test minutes | Per-minute, separate reporting fees | | Real mobile devices | Real Android phones, on demand | Emulation only | | Test private & staging environments | Built-in secure tunnel | Not available | | EU data residency | Yes, GDPR-compliant | US-region only | | Vendor history | Cross-browser cloud since 2012 | New product within Microsoft | ![Video, screenshots and logs out of the box](https://testingbot.com/assets/features/test-results-315382c39a67ad52167d2a4fd78570c99aea3e0ec386c17e82265851e93cb7b3.png) Test artifacts ## Video, screenshots and logs out of the box Every Playwright session ships with a full HD video, Playwright logs, browser console log and network HAR. Mark sessions as passed or failed from your test code, fetch artifacts via our [REST API](https://testingbot.com/support/api), and ship results back to Jira, Slack or your CI of choice. - Full HD video for every test - Built for speed and reliability - Console, network HAR and Playwright server logs - Custom screen resolution, timezone and locale - Browser startup flags and [many more options](https://testingbot.com/support/web-automate/playwright/options) Languages & runners ## Playwright integrations for every stack Playwright ships official client libraries for JavaScript, TypeScript, Python, Java and .NET. Drop them in alongside your test runner of choice. - ### [TypeScript / JavaScript](https://testingbot.com/support/web-automate/playwright/playwright-test) - [@playwright/test](https://testingbot.com/support/web-automate/playwright/playwright-test) - [Jest](https://testingbot.com/support/web-automate/playwright/jest) - [Mocha](https://testingbot.com/support/web-automate/playwright/mocha) - [CodeceptJS](https://testingbot.com/support/web-automate/playwright/codeceptjs) - ### [Python](https://testingbot.com/support/web-automate/playwright/pytest) - [PyTest](https://testingbot.com/support/web-automate/playwright/pytest) - [Robot Framework](https://testingbot.com/support/web-automate/playwright/robotframework) - ### [Java](https://testingbot.com/support/web-automate/playwright/java) - [Java quickstart](https://testingbot.com/support/web-automate/playwright/java) - ### [.NET](https://testingbot.com/support/web-automate/playwright/dotnet) - [.NET quickstart](https://testingbot.com/support/web-automate/playwright/dotnet) - ### [Ruby](https://testingbot.com/support/web-automate/playwright/ruby) - [Ruby quickstart](https://testingbot.com/support/web-automate/playwright/ruby) - ### [Recorder & extras](https://testingbot.com/support/web-automate/playwright/recorder) - [Recorder](https://testingbot.com/support/web-automate/playwright/recorder) - [Visual regression](https://testingbot.com/support/web-automate/playwright/visual-regression-testing) - [Performance](https://testingbot.com/support/web-automate/playwright/performance) Don't write tests manually? Use the [Playwright Recorder](https://testingbot.com/support/web-automate/playwright/recorder) to generate scripts and run them on the TestingBot grid. Massive parallelism ## Cut Playwright suite time from hours to minutes Fan out your Playwright workers across hundreds of single-use VMs running in parallel. Same test code, a fraction of the wall-clock time, no flakiness. More on how concurrency is sized and priced: [parallel testing](https://testingbot.com/features/parallel-testing). - Up to 100 concurrent Playwright sessions - Sessions start in seconds, no warm-up - Auto-retry, native Playwright --shard support and per-worker artifacts [Start free](https://testingbot.com/users/sign_up) ![Cut Playwright suite time from hours to minutes](https://testingbot.com/assets/features/selenium/parallel-0229d698e59a8a6b92fe50344a08634624f018331ecf8a849ef83cfb35e635a5.png) Live exit nodes 10+ countries United States United Kingdom Germany France Spain Belgium Australia China Japan Geolocation testing ## Test Playwright flows from 10+ countries Run Playwright tests from real exit nodes around the world. Verify **geo-targeting** , **geo-blocking** and **localisation** from the United States, the United Kingdom, Germany, France, Spain, Belgium, Australia, China, Japan and more. Secure infrastructure ## Real machines. Single-use. Zero leaks. Each Playwright session runs in a brand-new, single-use virtual machine on a real operating system. The VM is destroyed the moment the test ends, so nothing is reused and nothing leaks across customers. GDPR-compliant with EU data residency, SAML SSO and audit logs available for enterprise teams. - Single-use VMs, wiped after every session - GDPR-compliant, EU data residency available - SAML SSO, audit logs and SCIM provisioning [Visit TestingBot Trust Center](https://trust.testingbot.com/) 1. 1 Test connects to `cloud.testingbot.com` A pristine VM is provisioned in seconds, just for your session. 2. 2 Playwright drives a real browser Real Chrome, Firefox, Edge or WebKit, never simulated, never shared. 3. 3 VM destroyed the moment your test ends Cookies, files, browser state, all wiped. Nothing is ever reused. Continuous integration ## Plug Playwright into your CI/CD in minutes TestingBot integrates with every major CI/CD tool. Trigger Playwright suites on every commit and stream traces, videos and logs back into your build dashboards. ![Jenkins](https://testingbot.com/assets/integrations/jenkins-e586a1c4af833e4c8d38d7b392a4e84bd193f30cae9cf8c9f198a58ec943a1ff.webp) ### [Jenkins](https://testingbot.com/support/integrations/ci-cd/jenkins) ![Bamboo](https://testingbot.com/assets/integrations/bamboo-c8fe09a3cb0b74d6055e3aa6607d77720bfe29ff487b8e3cf825138adf634346.svg) ### [Bamboo](https://testingbot.com/support/integrations/ci-cd/bamboo) ![TeamCity](https://testingbot.com/assets/integrations/teamcity-7e75cdb835903a4983ad276c41d95e569ceac5be486e96ec1f955029ce5bb319.svg) ### [TeamCity](https://testingbot.com/support/integrations/ci-cd/teamcity) ![CircleCI](https://testingbot.com/assets/integrations/circleci-ede029654ff39b60342e4a5bba8fc9a3a49305cc38bda215b7cf8d07e85e0543.webp) ### [CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci) ![GitLab CI](https://testingbot.com/assets/integrations/gitlab-e4c0b6db07d07bc0a400f5606e258f4dc8a51b778e97bbfa51d1d531f3cb049a.webp) ### [GitLab CI](https://testingbot.com/support/integrations/ci-cd/gitlab) ![GitHub](https://testingbot.com/assets/integrations/github-actions-8436242ad9e15381f84f1a98a10dddce15611087f121d6305782e211942b7842.webp) ### [GitHub](https://testingbot.com/support/integrations/ci-cd/github-actions) Real mobile ## Playwright mobile testing on real Android devices Skip the emulators. Run Playwright mobile testing against Chrome on real Android devices in our cloud, with the same Playwright APIs, real hardware and real sensors. [View available devices](https://testingbot.com/support/web-automate/playwright/mobile) ![Playwright](https://testingbot.com/assets/support/playwright-2dcb4e75b55ac0d1135cfa47be6137b13f46a2047ce78b96a34f51b0e2cc0d96.svg) ![Android](https://testingbot.com/assets/environments/svg/android-1b3d235e17522e2238e1c55b4e9cdd41219f653cd958ea065877d3813f32ffc3.svg) Same Playwright APIs, real Android Chrome. Secure Tunnel ## Test internal & staging apps securely Use our [secure tunnel](https://testingbot.com/support/tunnel) to run Playwright tests against firewalled staging sites, internal dashboards or apps still on localhost. Encrypted end-to-end, no inbound firewall rules required, ready out of the box for popular CI providers. ![TestingBot secure tunnel for Playwright](https://testingbot.com/assets/features/tunnel-diagram-poster-8c25e19e6fe4f7276a46a96b31a0ad0e07bb4f5bf98dc192d2b70897cfbb1a26.png) [Read more](https://testingbot.com/support/tunnel) FAQ ## Playwright on TestingBot, frequently asked questions Why run Playwright on TestingBot instead of locally? Locally you are capped by your laptop or CI runner, usually 2–4 parallel browsers and only the OSes you have installed. TestingBot gives you up to 100 parallel Playwright sessions across real Chrome, Firefox, Edge and WebKit on real Windows, macOS and Linux machines, with the video and logs collected automatically. How is TestingBot different from Microsoft Playwright Testing (MPT)? TestingBot runs Playwright on real Chrome, Firefox, Edge and WebKit on real Windows, macOS and Linux. MPT runs Chromium on Linux only. TestingBot offers flat plans with unlimited test minutes, real Android devices and EU data residency. See [the comparison](https://testingbot.com#mpt) on this page. Which Playwright versions does TestingBot support? TestingBot supports every supported Playwright version, including the latest releases the day they ship. You can pin a specific Playwright version per-test using capabilities. Can I run Playwright tests in parallel? Yes. Plans on TestingBot include up to 100 parallel Playwright sessions. Each session runs in its own single-use VM, so tests do not interfere with one another. Does TestingBot provide test assets? Fully. Every session comes with a video recording, screenshots and logs. How do I migrate my existing Playwright tests? Set the `wsEndpoint` in your `playwright.config.ts` to `wss://cloud.testingbot.com/playwright`, add your TestingBot credentials, and pick a browser, OS or device through capabilities. See our [migration guide](https://testingbot.com/support/web-automate/playwright#update) for full examples. Can I test internal or staged sites? Yes, install our [secure tunnel](https://testingbot.com/support/tunnel) and your Playwright tests can reach staging environments, localhost or anything else behind your firewall, with no inbound rules required. Is the Playwright grid GDPR / SOC 2 compliant? TestingBot is not SOC 2 or ISO 27001 certified. We are GDPR-compliant with EU-only data residency, registered at CSA STAR Level 1 (a published self-assessment), and our control set is modelled on ISO 27001 Annex A. Our datacenter operators hold ISO 27001. We offer SAML SSO, SCIM and audit logs on enterprise plans. What is Playwright? Playwright is an open-source end-to-end testing framework built by Microsoft that drives Chromium, Firefox and WebKit through a single API. It supports JavaScript, TypeScript, Python, Java and .NET, with auto-waiting, network interception, fixtures and a built-in test runner. Teams use Playwright for cross-browser testing, visual regression testing, API testing, accessibility testing and mobile-web testing. TestingBot runs Playwright on real browsers, real operating systems and real Android devices in the cloud, so you can scale Playwright testing without managing infrastructure. Is Playwright free? Yes, Playwright itself is free. Microsoft releases it under the Apache 2.0 license, so you can install it, write tests and run them locally at no cost. The cost question for teams is usually how to run Playwright at scale across real browsers, devices and parallel workers, which is what TestingBot provides on flat plans with unlimited test minutes. Is Playwright open source? Playwright is open source. The framework is maintained by Microsoft on GitHub under the Apache 2.0 license, with a public roadmap and active community. TestingBot complements the open-source framework with a managed cloud grid for real-browser, real-device and parallel Playwright testing. Can Playwright be used for visual, accessibility and API testing? Yes. Playwright is a general-purpose automation framework, so the same scripts that drive end-to-end UI tests can also be used for visual regression testing, accessibility audits, API testing and performance checks. On TestingBot you get full HD video, Playwright traces, console and network HAR logs for every session, which makes visual and behaviour comparisons easier. Is TestingBot Playwright as a service? Yes. TestingBot is hosted Playwright, or Playwright as a service: you keep your own test code and config, and we run the browsers. There is no grid to provision, no images to patch and no capacity to plan for. Point @playwright/test at our endpoint and video, traces and logs come back with every run. Can I use TestingBot for Playwright automation in CI? Yes, and that is the common setup. Playwright automation runs the same way locally and in CI: the same config, credentials read from environment variables, and up to 100 concurrent sessions so a full suite finishes in minutes. Worked examples exist for GitHub Actions, GitLab CI, Jenkins, CircleCI and Azure Pipelines. ## Ready to scale your Playwright testing? Read our [Playwright Cloud Testing](https://testingbot.com/support/web-automate/playwright) documentation and run your first test in under 5 minutes. [Start a free trial](https://testingbot.com/users/sign_up) Resources you may like [![Visual Testing with Playwright](https://testingbot.com/assets/resources/articles/14-f3f2e8954c8a47cf0bb29059b44f4d2f40998442e22db6f8acdccb8e5b34d8c1.webp)](https://testingbot.com/resources/articles/playwright-visual-regression-testing) ### [Visual Testing with Playwright](https://testingbot.com/resources/articles/playwright-visual-regression-testing) Playwright provides automated browser testing. It offers a built-in feature to perform visual regression testing for your website. [Read more: Visual Testing with Playwright](https://testingbot.com/resources/articles/playwright-visual-regression-testing) [![Playwright Codegen: Record Tests in VS Code](https://testingbot.com/assets/resources/articles/32-277318b88c5e35eb2c96f03d9b53a1ef8eb2d96db184679892b5008b0bad9d0f.webp)](https://testingbot.com/resources/articles/playwright-recorder) ### [Playwright Codegen: Record Tests in VS Code](https://testingbot.com/resources/articles/playwright-recorder) How to record Playwright scripts from Visual Studio Code rather than writing them by hand, then run the recorded tests on a browser grid. [Read more: Playwright Codegen: Record Tests in VS Code](https://testingbot.com/resources/articles/playwright-recorder) --- URL: https://testingbot.com/features/automation/selenium Selenium WebDriver # Selenium Cloud Automation Run your Selenium WebDriver tests on 6100+ real browsers and physical iOS & Android devices. Massive parallelism, video & log artifacts, and battle-tested infrastructure with a 99.9% uptime SLA. Comparing frameworks? See [Playwright vs Selenium](https://testingbot.com/playwright-vs-selenium), [Cypress vs Selenium](https://testingbot.com/cypress-vs-selenium) or [Selenium vs Puppeteer](https://testingbot.com/selenium-vs-puppeteer). [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) - **Browsers & devices:** 6100+ - **Selenium versions:** All - **Uptime SLA:** 99.9% _Selenium tests running on Chrome, Firefox, Safari, Edge, Opera, Samsung, iOS and Android via TestingBot's cloud hub_ Trusted by some of the world's most innovative companies 5-line setup ## Point your existing tests at our hub One URL change is all it takes. Reuse your current Selenium WebDriver code, run it on our cloud, and pick any browser, OS or device through capabilities. hub.testingbot.com Copy JavaScript Python Java Ruby C# PHP ```javascript // Run a Selenium test on the TestingBot grid const wd = require('selenium-webdriver'); const driver = await new wd.Builder() .usingServer('https://hub.testingbot.com/wd/hub') .withCapabilities({ 'tb:options': { name: 'Checkout flow' }, browserName: 'chrome', browserVersion: 'latest', platformName: 'Windows 11' }) .build(); await driver.get('https://example.com'); await driver.quit(); ``` ```python # Run a Selenium test on the TestingBot grid from selenium import webdriver options = webdriver.ChromeOptions() options.set_capability('tb:options', {'name': 'Checkout flow'}) options.set_capability('browserVersion', 'latest') options.set_capability('platformName', 'Windows 11') driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options ) driver.get('https://example.com') driver.quit() ``` ```java // Run a Selenium test on the TestingBot grid DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("browserName", "chrome"); caps.setCapability("browserVersion", "latest"); caps.setCapability("platformName", "Windows 11"); WebDriver driver = new RemoteWebDriver( new URL("https://hub.testingbot.com/wd/hub"), caps ); driver.get("https://example.com"); driver.quit(); ``` ```ruby # Run a Selenium test on the TestingBot grid require 'selenium-webdriver' caps = Selenium::WebDriver::Remote::Capabilities.chrome( browserVersion: 'latest', platformName: 'Windows 11', 'tb:options' => { name: 'Checkout flow' } ) driver = Selenium::WebDriver.for(:remote, url: 'https://hub.testingbot.com/wd/hub', capabilities: caps ) driver.get 'https://example.com' driver.quit ``` ```csharp // Run a Selenium test on the TestingBot grid var options = new ChromeOptions(); options.AddAdditionalOption("browserVersion", "latest"); options.AddAdditionalOption("platformName", "Windows 11"); IWebDriver driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), options ); driver.Navigate().GoToUrl("https://example.com"); driver.Quit(); ``` ```php // Run a Selenium test on the TestingBot grid $capabilities = DesiredCapabilities::chrome(); $capabilities->setCapability('browserVersion', 'latest'); $capabilities->setCapability('platformName', 'Windows 11'); $driver = RemoteWebDriver::create( 'https://hub.testingbot.com/wd/hub', $capabilities ); $driver->get('https://example.com'); $driver->quit(); ``` Connected · [View documentation](https://testingbot.com/support/web-automate/selenium/nodejs) [View documentation](https://testingbot.com/support/web-automate/selenium/python) [View documentation](https://testingbot.com/support/web-automate/selenium/java) [View documentation](https://testingbot.com/support/web-automate/selenium/ruby) [View documentation](https://testingbot.com/support/web-automate/selenium/csharp) [View documentation](https://testingbot.com/support/web-automate/selenium/php) ![Rich test results out of the box](https://testingbot.com/assets/features/test-results-315382c39a67ad52167d2a4fd78570c99aea3e0ec386c17e82265851e93cb7b3.png) Test Results ## Rich test results out of the box Every Selenium session produces a full HD video, step-by-step command log, browser & Selenium logs and screenshots. Mark sessions as passed or failed from your test code, fetch artifacts via our [REST API](https://testingbot.com/support/api), and ship results back to Jira, Slack or your CI of choice. Every Selenium session is fully customizable, including: - All Selenium versions supported (3.x, 4.x, BiDi) - Custom screen resolution - Custom timezone - Change geolocation - Browser startup flags and [many more options](https://testingbot.com/support/web-automate/selenium/test-options) Languages & frameworks ## Selenium integrations for every stack TestingBot integrates with every major Selenium binding and test framework. Drop in your existing tests, no rewrites needed. - ### [Java](https://testingbot.com/support/web-automate/selenium/java) - [JUnit](https://testingbot.com/support/web-automate/selenium/java/junit) - [Parallel JUnit](https://testingbot.com/support/web-automate/selenium/java/parallel-junit) - [TestNG](https://testingbot.com/support/web-automate/selenium/java/testng) - [Selenide](https://testingbot.com/support/web-automate/selenium/java/selenide) - ### [Python](https://testingbot.com/support/web-automate/selenium/python) - [PyTest](https://testingbot.com/support/web-automate/selenium/python/pytest) - [PyUnit](https://testingbot.com/support/web-automate/selenium/python/pyunit) - [Behave](https://testingbot.com/support/web-automate/selenium/python/behave) - [Helium](https://testingbot.com/support/web-automate/selenium/python/helium) - ### [JavaScript](https://testingbot.com/support/web-automate/selenium/nodejs) - [WebDriverIO](https://testingbot.com/support/web-automate/selenium/nodejs/webdriverio) - [Nightwatch](https://testingbot.com/support/web-automate/selenium/nodejs/nightwatch) - [Protractor](https://testingbot.com/support/web-automate/selenium/nodejs/protractor) - [TestCafe](https://testingbot.com/support/web-automate/selenium/nodejs/testcafe) - ### [Ruby](https://testingbot.com/support/web-automate/selenium/ruby) - [RSpec](https://testingbot.com/support/web-automate/selenium/ruby/rspec) - [Cucumber](https://testingbot.com/support/web-automate/selenium/ruby/cucumber) - [Minitest](https://testingbot.com/support/web-automate/selenium/ruby/minitest) - [Test::Unit](https://testingbot.com/support/web-automate/selenium/ruby/testunit) - ### [C#](https://testingbot.com/support/web-automate/selenium/csharp) - [NUnit](https://testingbot.com/support/web-automate/selenium/csharp/nunit) - [MSTest](https://testingbot.com/support/web-automate/selenium/csharp/mstest) - [xUnit](https://testingbot.com/support/web-automate/selenium/csharp/xunit) - [SpecFlow](https://testingbot.com/support/web-automate/selenium/csharp/specflow) - ### [PHP](https://testingbot.com/support/web-automate/selenium/php) - [PHPUnit](https://testingbot.com/support/web-automate/selenium/php/phpunit) - [Codeception](https://testingbot.com/support/web-automate/selenium/php/codeception) - [Behat](https://testingbot.com/support/web-automate/selenium/php/behat-mink) - [SimpleTest](https://testingbot.com/support/web-automate/selenium/php/simpletest) Looking to run tests without writing code? Record tests with [Selenium IDE](https://testingbot.com/support/web-automate/codeless-automation/add-test) and upload them to our Codeless Automation for scheduled execution. Massive parallelism ## Cut suite time from hours to minutes Fan out your Selenium suite across hundreds of single-use VMs running in parallel. Same test code, a fraction of the wall-clock time, all on Windows, Linux and macOS optimised for speed and stability. More on how concurrency is sized and priced: [parallel testing](https://testingbot.com/features/parallel-testing). - Up to 100 concurrent Selenium sessions - No queue, no warm-up. Sessions start in seconds - Auto-retry, and your test runner's own parallelism works unchanged [Start free](https://testingbot.com/users/sign_up) ![Cut suite time from hours to minutes](https://testingbot.com/assets/features/selenium/parallel-0229d698e59a8a6b92fe50344a08634624f018331ecf8a849ef83cfb35e635a5.png) Live exit nodes 10+ countries United States United Kingdom Germany France Spain Belgium Australia China Japan Geolocation testing ## Test from 10+ countries Run Selenium tests from real exit nodes around the world. Verify **geo-targeting** , **geo-blocking** and **localisation** from Australia, Belgium, the United Kingdom, the United States, Germany, France, Spain, China and more. Secure infrastructure ## Real machines. Single-use. Zero leaks. Each Selenium session runs in a brand-new, single-use virtual machine on a real operating system. The VM is destroyed the moment the test ends, nothing is reused, nothing leaks across customers. GDPR-compliant, with SAML SSO and audit logs for teams. - Single-use VMs, wiped after every session - GDPR-compliant, EU data residency available - SAML SSO, audit logs and SCIM provisioning [Visit TestingBot Trust Center](https://trust.testingbot.com/) 1. 1 Test connects to `hub.testingbot.com` A pristine VM is provisioned in seconds, just for your session. 2. 2 Your Selenium commands run on real hardware Real Windows, macOS or Linux — never simulated, never shared. 3. 3 VM destroyed the moment your test ends Cookies, files, browser state — all wiped. Nothing is ever reused. Continuous integration ## Plug into your CI/CD in minutes TestingBot integrates with every major CI/CD tool. Trigger Selenium suites on every commit and stream results back into your build dashboards. ![Jenkins](https://testingbot.com/assets/integrations/jenkins-e586a1c4af833e4c8d38d7b392a4e84bd193f30cae9cf8c9f198a58ec943a1ff.webp) ### [Jenkins](https://testingbot.com/support/integrations/ci-cd/jenkins) ![Bamboo](https://testingbot.com/assets/integrations/bamboo-c8fe09a3cb0b74d6055e3aa6607d77720bfe29ff487b8e3cf825138adf634346.svg) ### [Bamboo](https://testingbot.com/support/integrations/ci-cd/bamboo) ![TeamCity](https://testingbot.com/assets/integrations/teamcity-7e75cdb835903a4983ad276c41d95e569ceac5be486e96ec1f955029ce5bb319.svg) ### [TeamCity](https://testingbot.com/support/integrations/ci-cd/teamcity) ![CircleCI](https://testingbot.com/assets/integrations/circleci-ede029654ff39b60342e4a5bba8fc9a3a49305cc38bda215b7cf8d07e85e0543.webp) ### [CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci) ![GitLab CI](https://testingbot.com/assets/integrations/gitlab-e4c0b6db07d07bc0a400f5606e258f4dc8a51b778e97bbfa51d1d531f3cb049a.webp) ### [GitLab CI](https://testingbot.com/support/integrations/ci-cd/gitlab) ![GitHub](https://testingbot.com/assets/integrations/github-actions-8436242ad9e15381f84f1a98a10dddce15611087f121d6305782e211942b7842.webp) ### [GitHub](https://testingbot.com/support/integrations/ci-cd/github-actions) Mobile Selenium ## Mobile web Selenium on real devices Drive Mobile Safari on real iPhones and iPads, and Chrome on real Android devices using the same Selenium WebDriver protocol. Real hardware, real sensors, accurate results. [View all devices](https://testingbot.com/support/app-automate/devices) ![Test online on Pixel 8](https://testingbot.com/assets/devices/160h/46-d0ca4c1d1985aefc27c52ba2a1c88748875c574d5358e53adcbdeaee4ed078bb.webp) Pixel 8 ![Test online on Pixel 11](https://testingbot.com/assets/devices/160h/45-b90994534fadde53be99a94521c6ea23a39fdf32ad277240be4e2613540b5019.webp) Pixel 11 ![Test online on iPhone 17 Pro](https://testingbot.com/assets/devices/160h/44-6b4402c6996d9b1620879bb1123c40a6b36abb064e65dca0a1b3f05286779e9c.webp) iPhone 17 Pro ![Test online on iPhone 17 Pro Max](https://testingbot.com/assets/devices/160h/43-59d7e30684332ed4bf25567cbc09ca942eac46d925ccde8352d3e85812480f7d.webp) iPhone 17 Pro Max ![Test online on iPhone 16 Pro](https://testingbot.com/assets/devices/160h/42-de3704079345026847b739a5b1cf5332595904e54ede66abd988a76fb1e6593c.webp) iPhone 16 Pro ![Test online on iPad (9th generation)](https://testingbot.com/assets/devices/160h/41-70f7d18afde7620063a49a76cbfd088f8b59c742f3a5f4dd4a513dd6ec5c676c.webp) iPad (9th generation) Secure Tunnel ## Test internal & staging apps securely Use our [secure tunnel](https://testingbot.com/support/tunnel) to run Selenium tests against firewalled staging sites, internal dashboards or apps still on localhost. Encrypted end-to-end, no inbound firewall rules required, and built-in support for popular CI providers. ![TestingBot secure tunnel for Selenium](https://testingbot.com/assets/features/tunnel-diagram-poster-8c25e19e6fe4f7276a46a96b31a0ad0e07bb4f5bf98dc192d2b70897cfbb1a26.png) [Read more](https://testingbot.com/support/tunnel) Buy vs build ## TestingBot vs a self-hosted Selenium grid Running your own Selenium grid sounds cheap until you account for the maintenance, the flakiness and the missing devices. Here is how a hosted Selenium cloud compares. | Capability | TestingBot | Self-hosted Selenium Grid | | --- | --- | --- | | Browser coverage | 6100+ browsers, all major versions, refreshed weekly | Limited to what you provision and patch yourself | | Time to first test | Minutes, change one URL in your code | Days to weeks of grid setup and tuning | | Real iOS & Android | Real physical devices on demand | Requires a separate device lab | | Parallel scaling | Up to 100 concurrent sessions, on demand | Capped by your hardware budget | | Video & logs | Video, network, browser & Selenium logs included | Roll your own recording & log aggregation | | Reliability | 99.9% uptime SLA, 24/7 monitoring | You are the on-call engineer | | Support | 24/7 human support and dedicated CSM | Stack Overflow and your own logs | FAQ ## Selenium on TestingBot: frequently asked questions What is a cloud-based Selenium grid? A cloud-based Selenium grid is a managed pool of browser nodes that you connect to via the `https://hub.testingbot.com/wd/hub` endpoint. Your existing Selenium WebDriver code stays the same, only the remote URL and a few capabilities change. TestingBot handles the OS images, browser installs, scaling and recording for you. Which Selenium versions does TestingBot support? TestingBot supports every Selenium version still in active use, from Selenium 2.x and 3.x through Selenium 4 and Selenium BiDi. You can pin a specific server version per-test using the `se:options` or `tb:options` capability. Can I run Selenium tests in parallel? Yes. Plans on TestingBot include up to 100 parallel Selenium sessions. Each session runs in its own single-use VM, so tests do not interfere with one another and parallelism does not introduce flakiness. Does TestingBot work with Selenium 4 and BiDi? Fully. Our hub speaks Selenium 4 W3C WebDriver and supports the new [Selenium BiDi protocol](https://testingbot.com/support/web-automate/selenium/selenium-bidi) for advanced features like real-time logs, network interception and authentication handling. How do I migrate existing Selenium tests to TestingBot? Change the remote URL to `https://hub.testingbot.com/wd/hub`, add your TestingBot credentials, and pick a browser/OS via capabilities. See our [migration guide](https://testingbot.com/support/web-automate/selenium/migrate-existing-tests) for step-by-step examples in every language. Can I test internal or staged sites? Yes, install our [secure tunnel](https://testingbot.com/support/tunnel) and your Selenium tests can reach staging environments, localhost servers or anything else behind your firewall, with no inbound rules required. Does TestingBot record videos and logs of my Selenium tests? Every Selenium session is recorded as full HD video alongside step-by-step Selenium commands, browser console logs, network HAR files and screenshots. Artifacts are accessible from the dashboard or via the REST API for CI integrations. Is the Selenium grid SOC 2 / GDPR compliant? TestingBot is not SOC 2 or ISO 27001 certified. We are GDPR-compliant with EU-only data residency, registered at CSA STAR Level 1 (a published self-assessment), and our control set is modelled on ISO 27001 Annex A. Our datacenter operators hold ISO 27001. We offer SAML SSO, SCIM and audit logs on enterprise plans. Do I still need to run my own Selenium Grid? No. A cloud Selenium Grid gives you the same hub-and-node model without any hub, nodes or driver upgrades to maintain. You point your existing WebDriver code at our endpoint, and we handle browser installs, OS images, scaling and cleanup between sessions. Is TestingBot a Selenium testing tool or hosted infrastructure? Both. As a Selenium testing tool it adds video, console and network artifacts, live debugging and a REST API on top of plain WebDriver. As infrastructure it runs the grid itself, so the tooling and the machines come from one place. ## More reasons to love TestingBot ### 24/7 support Got questions? Ask in the 24/7 in-app customer [chat support](https://testingbot.com#) or [send us an email](https://testingbot.com/contact/new). We are here to help you! TestingBot Support Hey 👋 How can we help? I need support ### Trusted by over 14,000 teams 6M+ Tests 120k+ Users 400+ Enterprises 106 Countries [Request a demo](https://testingbot.com/demo) ### Developer Documentation Everything you need to integrate TestingBot into your workflow. Browse guides, API references and SDKs for web and mobile testing. - [Selenium Quickstart Documentation](https://testingbot.com/support/web-automate/selenium) - [Selenium 4](https://testingbot.com/support/web-automate/selenium/selenium4) - [Selenium BiDi](https://testingbot.com/support/web-automate/selenium/selenium-bidi) - [CI/CD integration](https://testingbot.com/support/integrations/ci-cd) [Explore Documentation](https://testingbot.com/support) ### Enterprise-grade Security STAR Level 1 Certified GDPR Compliant SAML SSO Single Sign-On 99.9% Uptime SLA [Visit TestingBot Trust Center](https://trust.testingbot.com/) ## Ready to scale your Selenium testing? Read our [Selenium Cloud Testing](https://testingbot.com/support/web-automate/selenium) documentation and run your first test in under 5 minutes. [Start a free trial](https://testingbot.com/users/sign_up) Resources you may like [![Selenium and Generative AI](https://testingbot.com/assets/resources/articles/28-af707bf052844eb00f861bb6ebba57e6be4479dde1c78a5df4e70d2f597a8b18.webp)](https://testingbot.com/resources/articles/generative-ai-selenium) ### [Selenium and Generative AI](https://testingbot.com/resources/articles/generative-ai-selenium) How to use generative AI to produce realistic test data for Selenium runs, and why varied data uncovers bugs that fixed fixtures never will. [Read more: Selenium and Generative AI](https://testingbot.com/resources/articles/generative-ai-selenium) [![Selenium 4: what's new](https://testingbot.com/assets/blog/2021/se4.webp)](https://testingbot.com/blog/selenium-4) ### [Selenium 4: what's new](https://testingbot.com/blog/selenium-4) Learn more about what is new Selenium 4. [Read more: Selenium 4: what's new](https://testingbot.com/blog/selenium-4) [![Selenium WebDriver BiDi](https://testingbot.com/assets/blog/2024/bidi.webp)](https://testingbot.com/blog/webdriver-bidi) ### [Selenium WebDriver BiDi](https://testingbot.com/blog/webdriver-bidi) WebDriver BiDi support is now available on TestingBot. Learn how to use this exciting new improvement with your Selenium tests. [Read more: Selenium WebDriver BiDi](https://testingbot.com/blog/webdriver-bidi) --- URL: https://testingbot.com/features/automation/cypress Cypress cloud # Run Cypress on a real browser cloud Point your existing Cypress suite at TestingBot's grid and run it across real Chrome, Firefox and Edge on Windows, macOS and Linux. Massive parallelism, video & log artifacts, GDPR-compliant with EU data residency. Comparing frameworks? See [Cypress vs Selenium](https://testingbot.com/cypress-vs-selenium) and [Cypress vs Playwright](https://testingbot.com/cypress-vs-playwright). [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) - **Browsers & OSes:** Chrome · Firefox · Edge - **Parallel sessions:** 100 - **Uptime SLA:** 99.9% Trusted by some of the world's most innovative companies 30-second answer ## Why run Cypress on TestingBot? Real cross-browser coverage Cypress runs on real Chrome, Firefox and Edge in our cloud, on real Windows, macOS and Linux. Same specs, every browser your customers use. Massive parallelism Fan out to up to 100 concurrent Cypress sessions, each in its own single-use VM. Cut suite time from hours to minutes. Drop-in CLI swap Install our open-source `testingbot-cypress-cli`, point it at your `cypress.config.js`, and run. What is Cypress? ## Cypress in 60 seconds ![Cypress](https://testingbot.com/assets/support/cypress-409c9fd3f79115ed7676aa1df4f6a217dc4402b3366a61984924aec7b4fefa97.svg) ### Cypress Open source · MIT · JavaScript / TypeScript Cypress is a JavaScript end-to-end testing framework that runs in the browser, in the same JavaScript runtime as your app. Tests get direct access to every object, the DOM, network requests, timers and the framework's own state, without the cross-process bridge that WebDriver-based tools use. That gives Cypress its trademark fast feedback loop, automatic waiting, time-travel debugging and built-in spy / stub APIs. It is the canonical choice for JavaScript-first teams that want a fast feedback loop and modern DX. - JavaScript / TypeScript-first, runs in-browser - Real Chrome, Firefox and Edge - Auto-wait, time-travel debugger, network stubbing - Built-in spy, stub and clock APIs - Parallel via the testingbot-cypress-cli Two-line CLI ## Run Cypress on the cloud with one command Install our open-source CLI, run testingbot-cypress init to generate a config, then testingbot-cypress run to fan out across real browsers in parallel. install & init step 1 ```bash # Install the open-source CLI $ npm install testingbot-cypress-cli # Generate testingbot.json (one-off) $ testingbot-cypress init # Edit the generated testingbot.json: # "cypress_project_dir": "./cypress", # "browsers": ["chrome", "firefox", "edge"], # "build_name": "ci-build-${BUILD_NUMBER}" ``` run on the cloud step 2 ```bash # Run your specs on real cloud browsers $ testingbot-cypress run --parallel 10 # Pin a specific supported Cypress version $ testingbot-cypress run \ --cypress-version "14.2.0" # → live results in your TestingBot dashboard # → video and console logs via REST API ``` See the full [Cypress on TestingBot documentation](https://testingbot.com/support/web-automate/cypress) for advanced flags, parallel options and CI integration. Local vs cloud ## Local Cypress vs TestingBot Cypress is fast on a laptop until you need scale, multiple browsers or real WebKit. Here is how a hosted cloud compares. | Capability | TestingBot | Local Cypress | | --- | --- | --- | | Browser coverage | Real Chrome, Firefox and Edge | Whatever you have installed locally | | Operating systems | Real Windows, macOS and Linux | Your single dev machine | | Parallel scaling | Up to 100 concurrent sessions, on demand | Capped by your CPU and RAM | | Video & logs | HD video and console logs included | Built-in video, but no central dashboard | | Test private & staging environments | Built-in secure tunnel | Direct access only | | EU data residency | Yes, GDPR-compliant | Whatever country your laptop is in | | Reliability | 99.9% uptime SLA, 24/7 monitoring | Crashes when you close the lid | Cypress Cloud alternative ## Looking for a Cypress Cloud alternative? Cypress Cloud, formerly the Cypress Dashboard, records and orchestrates runs on machines you provide. TestingBot provides the machines too: the same parallel runs, video and logs, plus the browsers and operating systems to run them on. | What you are comparing | TestingBot | Cypress Cloud | | --- | --- | --- | | What it provides | Browsers, operating systems and the recording | Recording, orchestration and analytics | | Where your tests run | On our single-use VMs, wiped after every run | On CI runners you supply and maintain | | How it is billed | By the parallel sessions on your plan | Per recorded test result | | Cross-browser coverage | Chrome, Firefox, Edge and WebKit on Windows and macOS | Whatever your own runners have installed | ### When Cypress Cloud is the better fit If you already have all the CI capacity you need, and what you want is Test Replay, flake detection and Cypress-specific analytics layered on top of it, Cypress Cloud is built for exactly that and we do not try to replicate it. Teams tend to come to us when the constraint is machines and browser coverage rather than reporting. Cypress Cloud details from cypress.io/pricing, checked 20 August 2026. ![Video, screenshots and logs out of the box](https://testingbot.com/assets/features/test-results-315382c39a67ad52167d2a4fd78570c99aea3e0ec386c17e82265851e93cb7b3.png) Test artifacts ## Video, screenshots and logs out of the box Every Cypress run ships with a full HD video, browser console log and screenshots. Inspect a failure without re-running the suite. Mark sessions as passed or failed from your tests, fetch artifacts via our [REST API](https://testingbot.com/support/api), and ship results back to Jira, Slack or your CI of choice. - Full HD video for every spec run - Browser console output and Cypress logs - Screenshots on demand - Custom screen resolution, timezone and locale - Browser startup flags and [many more capabilities](https://testingbot.com/support/web-automate/cypress/capabilities) Capabilities ## Everything Cypress needs in the cloud Beyond running tests, TestingBot exposes the full set of Cypress capabilities you would set in your config, plus cloud-only features like geolocation and timezone. - ### [Capabilities](https://testingbot.com/support/web-automate/cypress/capabilities) Browser, OS, version, screen resolution and every capability the cloud accepts. - ### [Authentication](https://testingbot.com/support/web-automate/cypress/authentication) Pass credentials, environment variables and login fixtures into your specs. - ### [Specs & filters](https://testingbot.com/support/web-automate/cypress/specs) Pick which specs run per build, group by tag or shard automatically. - ### [TypeScript](https://testingbot.com/support/web-automate/cypress/typescript) Run TypeScript-based Cypress specs out of the box, no extra setup. - ### [Geolocation & timezone](https://testingbot.com/support/web-automate/cypress/geolocation) Spoof GPS and timezone per session for App Store, store-front and back-end checks. - ### [Reporting](https://testingbot.com/support/web-automate/cypress/reporting) JUnit XML, mochawesome and integrations with Jenkins, GitHub and GitLab. Massive parallelism ## Cut Cypress suite time from hours to minutes Run hundreds of Cypress sessions in parallel across single-use VMs. Same specs, a fraction of the wall-clock time, with smart spec sharding handled by the CLI. More on how concurrency is sized and priced: [parallel testing](https://testingbot.com/features/parallel-testing). - Up to 100 concurrent Cypress sessions - Sessions start in seconds, no warm-up - Smart spec sharding via testingbot-cypress-cli [Start free](https://testingbot.com/users/sign_up) ![Cut Cypress suite time from hours to minutes](https://testingbot.com/assets/features/selenium/parallel-0229d698e59a8a6b92fe50344a08634624f018331ecf8a849ef83cfb35e635a5.png) Live exit nodes 10+ countries United States United Kingdom Germany France Spain Belgium Australia China Japan Geolocation testing ## Test Cypress flows from 10+ countries Run Cypress tests from real exit nodes around the world. Verify **geo-targeting** , **geo-blocking** and **localisation** from the United States, the United Kingdom, Germany, France, Spain, Belgium, Australia, China, Japan and more. Secure infrastructure ## Real machines. Single-use. Zero leaks. Each Cypress session runs in a brand-new, single-use virtual machine on a real operating system. The VM is destroyed the moment your run ends, nothing is reused, nothing leaks across customers. GDPR-compliant with EU data residency, SAML SSO and audit logs available for enterprise teams. - Single-use VMs, wiped after every run - GDPR-compliant, EU data residency available - SAML SSO, audit logs and SCIM provisioning [Visit TestingBot Trust Center](https://trust.testingbot.com/) 1. 1 Test connects to the TestingBot grid A pristine VM is provisioned in seconds, just for your session. 2. 2 Cypress drives a real browser Real Chrome, Firefox or Edge, never simulated, never shared. 3. 3 VM destroyed the moment your run ends Cookies, files, browser state, all wiped. Nothing is ever reused. Continuous integration ## Plug Cypress into your CI/CD in minutes TestingBot integrates with every major CI/CD tool. Trigger Cypress runs on every commit and stream videos and logs back into your build dashboards. ![Jenkins](https://testingbot.com/assets/integrations/jenkins-e586a1c4af833e4c8d38d7b392a4e84bd193f30cae9cf8c9f198a58ec943a1ff.webp) ### [Jenkins](https://testingbot.com/support/integrations/ci-cd/jenkins) ![Bamboo](https://testingbot.com/assets/integrations/bamboo-c8fe09a3cb0b74d6055e3aa6607d77720bfe29ff487b8e3cf825138adf634346.svg) ### [Bamboo](https://testingbot.com/support/integrations/ci-cd/bamboo) ![TeamCity](https://testingbot.com/assets/integrations/teamcity-7e75cdb835903a4983ad276c41d95e569ceac5be486e96ec1f955029ce5bb319.svg) ### [TeamCity](https://testingbot.com/support/integrations/ci-cd/teamcity) ![CircleCI](https://testingbot.com/assets/integrations/circleci-ede029654ff39b60342e4a5bba8fc9a3a49305cc38bda215b7cf8d07e85e0543.webp) ### [CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci) ![GitLab CI](https://testingbot.com/assets/integrations/gitlab-e4c0b6db07d07bc0a400f5606e258f4dc8a51b778e97bbfa51d1d531f3cb049a.webp) ### [GitLab CI](https://testingbot.com/support/integrations/ci-cd/gitlab) ![GitHub](https://testingbot.com/assets/integrations/github-actions-8436242ad9e15381f84f1a98a10dddce15611087f121d6305782e211942b7842.webp) ### [GitHub](https://testingbot.com/support/integrations/ci-cd/github-actions) Secure Tunnel ## Test internal & staging apps securely Use our [secure tunnel](https://testingbot.com/support/tunnel) to run Cypress tests against firewalled staging sites, internal dashboards or apps still on localhost. Encrypted end-to-end, no inbound firewall rules required, ready out of the box for popular CI providers. ![TestingBot secure tunnel for Cypress](https://testingbot.com/assets/features/tunnel-diagram-poster-8c25e19e6fe4f7276a46a96b31a0ad0e07bb4f5bf98dc192d2b70897cfbb1a26.png) [Read more](https://testingbot.com/support/tunnel) FAQ ## Cypress on TestingBot: frequently asked questions Why run Cypress on TestingBot instead of locally? Locally Cypress runs on whatever browser you have installed. TestingBot gives you up to 100 parallel Cypress sessions on real Chrome, Firefox and Edge across real Windows, macOS and Linux machines, with HD video and console logs collected automatically. Which Cypress versions are supported? TestingBot currently supports Cypress `14.2.0`, `13.17.0`, `13.3.3` (default), `12.5.1`, `10.0.0`, `9.1.0` and `8.0.0`. Pin a specific version per-run via the `cypress_version` field in your `testingbot.json`. Can I run Cypress in parallel? Yes. Plans on TestingBot include up to 100 parallel Cypress sessions, with automatic spec sharding handled by the testingbot-cypress-cli. Each session runs in its own single-use VM. How is TestingBot different from Cypress Cloud? TestingBot runs Cypress on real Chrome, Firefox and Edge across real Windows, macOS and Linux. Cypress Cloud is a results dashboard, not a browser cloud. The two can coexist, many teams record to Cypress Cloud and execute on TestingBot. Does TestingBot support TypeScript and the Cypress component runner? Yes, see the [TypeScript guide](https://testingbot.com/support/web-automate/cypress/typescript). Component runs work the same way as e2e specs. How do I migrate my existing Cypress suite? Install our open-source `testingbot-cypress-cli`, run `testingbot-cypress init`, point it at your `cypress.config.js`, and run `testingbot-cypress run`. Your specs, plugins and fixtures stay unchanged. Can I test internal or staged sites? Yes, install our [secure tunnel](https://testingbot.com/support/tunnel) and your Cypress tests can reach staging environments, localhost or anything else behind your firewall. Is the Cypress grid GDPR / SOC 2 compliant? TestingBot is not SOC 2 or ISO 27001 certified. We are GDPR-compliant with EU-only data residency, registered at CSA STAR Level 1 (a published self-assessment), and our control set is modelled on ISO 27001 Annex A. Our datacenter operators hold ISO 27001. We offer SAML SSO, SCIM and audit logs on enterprise plans. Is TestingBot a Cypress Cloud alternative? For the parts most teams pay for, yes. TestingBot runs your specs in parallel, records video and logs and gives you a dashboard, and it supplies the browsers and machines as well, which Cypress Cloud does not. It is billed by parallel sessions rather than per recorded test result. If you specifically want Test Replay or Cypress-native flake analytics on your own CI, Cypress Cloud remains the right tool. Which browsers can I run Cypress on? Chrome, Chromium, Firefox, Edge, Electron and WebKit, on Windows and macOS. That covers cross-browser Cypress testing you cannot easily get on a single CI runner, including WebKit for Safari-like rendering, and headless or headed runs of any of them. ## Ready to scale your Cypress testing? Read our [Cypress Cloud Testing](https://testingbot.com/support/web-automate/cypress) documentation and run your first spec in under 5 minutes. [Start a free trial](https://testingbot.com/users/sign_up) Resources you may like [![Tutorial on debugging Cypress tests](https://testingbot.com/assets/resources/articles/10-7ae9f6d9435bea1c54ec815ad535dc4a590086e6165263a9eba3441635d55726.webp)](https://testingbot.com/resources/articles/how-to-debug-cypress-tests) ### [Tutorial on debugging Cypress tests](https://testingbot.com/resources/articles/how-to-debug-cypress-tests) How to debug a failing Cypress test using the debugger, console logs, the stack trace, and the debug and pause commands. [Read more: Tutorial on debugging Cypress tests](https://testingbot.com/resources/articles/how-to-debug-cypress-tests) [![Cypress and Cucumber Browser Testing](https://testingbot.com/assets/resources/articles/6-495c5e14cb4bf1e282858fc7e2cfcbe89775e3790b88d34c15bf9a6f36d387d4.webp)](https://testingbot.com/resources/articles/cypress-cucumber-testing) ### [Cypress and Cucumber Browser Testing](https://testingbot.com/resources/articles/cypress-cucumber-testing) How to combine Cypress with Cucumber so end-to-end tests read as plain language, including a worked example and the TypeScript setup. [Read more: Cypress and Cucumber Browser Testing](https://testingbot.com/resources/articles/cypress-cucumber-testing) [![Run Cypress tests on TestingBot's Browser Grid](https://testingbot.com/assets/blog/2020/cypress.webp)](https://testingbot.com/blog/testingbot-cypress) ### [Run Cypress tests on TestingBot's Browser Grid](https://testingbot.com/blog/testingbot-cypress) Run Cypress Automated tests on TestingBot's Browser grid. [Read more: Run Cypress tests on TestingBot's Browser Grid](https://testingbot.com/blog/testingbot-cypress) --- URL: https://testingbot.com/features/automation/espresso Espresso cloud # Run Espresso on real Android devices Run your Espresso suite on real Galaxy, Pixel, OnePlus and Xiaomi devices from Android 4.4 through Android 15 with a single CLI command. Massive parallelism, video & log artifacts and a 99.9% uptime SLA. Choosing a framework? See [Appium vs Espresso](https://testingbot.com/appium-vs-espresso), or the [full automation framework lineup](https://testingbot.com/features). [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) - **Real Android devices:** 200+ - **Android versions:** 4.4 → 15 - **Uptime SLA:** 99.9% Trusted by some of the world's most innovative companies 30-second answer ## Why run Espresso on TestingBot? Real Android devices Run on a fleet of real Galaxy, Pixel, OnePlus and Xiaomi devices, not just emulators. Latest flagships on day one, plus a deep legacy fleet from Android 4.4. Replace your in-house Android lab Skip the cabinet of dusty phones, the swollen batteries and the OEM update treadmill. TestingBot has run Android tests on real devices for 5+ years. Native Java & Kotlin Use the same Espresso framework Google ships with Android Studio. Same APIs, same JUnit assertions, real native performance. What is Espresso? ## Espresso in 60 seconds ![Espresso](https://testingbot.com/assets/environments/svg/android-1b3d235e17522e2238e1c55b4e9cdd41219f653cd958ea065877d3813f32ffc3.svg) ### Espresso Google · built into Android Studio · Java / Kotlin Espresso is Google's official UI testing framework for Android, built into Android Studio and the Android SDK. It uses the AndroidJUnitRunner instrumentation runner, and lets you script taps, swipes, scrolls, intents and view assertions against your app's real UI. Because Espresso runs in the same process as your app, tests are fast and reliable, with automatic synchronisation between the test and the UI thread. It is the canonical choice for fast, in-process Android UI tests. - Native Google framework, built into Android Studio - Java and Kotlin bindings - Real Galaxy, Pixel, OnePlus, Xiaomi and emulators - View matchers, view actions, view assertions, intents - AndroidJUnitRunner, JUnit XML output for CI One-line CLI ## Run Espresso on real devices with one command Install the TestingBot CLI, point it at your app APK and your test APK, pick a device — done. Uploads, capabilities and result polling are all handled for you. install the CLI step 1 ```bash # Install the TestingBot CLI $ npm install -g testingbot-cli # Authenticate (once) — set your KEY and SECRET $ export TESTINGBOT_KEY=your-key $ export TESTINGBOT_SECRET=your-secret # Build app + Espresso test APKs (one-off) $ ./gradlew assembleDebug \ assembleDebugAndroidTest ``` run Espresso step 2 ```bash # Run Espresso on a real Pixel — one command $ testingbot espresso app.apk app-test.apk \ --device "Pixel 8" \ --platform-version "14" \ --real-device # Or, on a Galaxy $ testingbot espresso app.apk app-test.apk \ --device "Galaxy S24" \ --real-device # → live results in your TestingBot dashboard # → JUnit XML, video and logcat via REST API ``` See the full [Espresso on TestingBot documentation](https://testingbot.com/support/app-automate/espresso) for advanced flags, parallel runs and CI integration. Buy vs build ## TestingBot vs an in-house Android device lab Maintaining your own Android device lab is expensive. Devices break, OEMs ship updates on their own schedule, and Samsung's Knox controls keep changing. Here is how a hosted Android device cloud compares. | Capability | TestingBot | In-house Android lab | | --- | --- | --- | | Device coverage | Galaxy, Pixel, OnePlus, Xiaomi from Android 4.4 to 15 | Limited to what you can buy and rack | | Time to first test | Minutes, one CLI command and you are running | Weeks of provisioning, ADB, MDM and tethering | | Parallel scaling | Up to 100 concurrent Espresso sessions | Capped by the number of devices you own | | Video & logs | Video, logcat, screenshots and JUnit XML | Roll your own recording & log aggregation | | Reliability | 99.9% uptime SLA, 24/7 monitoring | You replace dead batteries and broken screens | | Security | Devices wiped between every session | Manual cleanup, leftover state risks | | Support | 24/7 human support and dedicated CSM | Stack Overflow and your own logs | Capabilities ## Everything Espresso needs in the cloud Beyond running tests, TestingBot exposes the full set of Espresso capabilities you would set in Android Studio, plus cloud-only features like geolocation and network throttling. - ### [Instrumentation](https://testingbot.com/support/app-automate/espresso/instrumentation) Configure AndroidJUnitRunner instrumentation arguments per run. - ### [Test filters](https://testingbot.com/support/app-automate/espresso/filter) Run a subset of your Espresso suite without rebuilding the bundle. - ### [Test reports](https://testingbot.com/support/app-automate/espresso/test-reports) Video, logcat, screenshots and JUnit XML for every run. - ### [Screenshots](https://testingbot.com/support/app-automate/espresso/screenshots) Take in-test screenshots and pull them via the REST API. - ### [Network throttling](https://testingbot.com/support/app-automate/espresso/simulate-network-conditions) Simulate 3G, 4G, lossy WiFi and offline conditions per session. - ### [Localization & geo](https://testingbot.com/support/app-automate/espresso/set-localization-options) Set device locale, language and GPS coordinates for store-front and back-end checks. Real Android devices ## Galaxy, Pixel, OnePlus & Xiaomi on demand A fleet of real Android devices spanning Android 4.4 through Android 15, refreshed every release. Latest flagship phones available the day OEMs ship them. [View all devices](https://testingbot.com/support/app-automate/devices) ![Test online on Pixel 8](https://testingbot.com/assets/devices/160h/46-d0ca4c1d1985aefc27c52ba2a1c88748875c574d5358e53adcbdeaee4ed078bb.webp) Pixel 8 ![Test online on Pixel 11](https://testingbot.com/assets/devices/160h/45-b90994534fadde53be99a94521c6ea23a39fdf32ad277240be4e2613540b5019.webp) Pixel 11 ![Test online on Galaxy S26](https://testingbot.com/assets/devices/160h/40-e917ed08e15b5941fed75855ac457dd1df46b471a2020f081c4d70cefb68f4e8.webp) Galaxy S26 ![Test online on Huawei P40](https://testingbot.com/assets/devices/160h/39-32a788c7e66a3da1c88eaaefefd37381f729e510f096b43906968b6907c44c57.webp) Huawei P40 ![Test online on Pixel 10](https://testingbot.com/assets/devices/160h/37-738ed465eff0a4bcc33a5294f6f9f4d701fed317ce0e97e202ffc3f9dca7edd4.webp) Pixel 10 ![Test online on Pixel 9](https://testingbot.com/assets/devices/160h/35-abb9a661ebbfa84f18bd6985d3e80330ad43fe4ea7d408157c87ae0f560a5ca1.webp) Pixel 9 ![Video, screenshots and JUnit XML for every run](https://testingbot.com/assets/features/test-results-315382c39a67ad52167d2a4fd78570c99aea3e0ec386c17e82265851e93cb7b3.png) Test reports ## Video, screenshots and JUnit XML for every run Every Espresso run ships with a video screencast, logcat output, viewport screenshots and a JUnit XML report compatible with every major CI parser. Mark sessions as passed or failed from your tests, fetch artifacts via our [REST API](https://testingbot.com/support/api), and ship results into Jira, Slack or your CI dashboard. - Full HD video for every test run - logcat output, app logs and crash reports - Viewport screenshots on demand - JUnit XML for Jenkins, GitHub Actions, GitLab CI Continuous integration ## Plug Espresso into your CI/CD in minutes TestingBot integrates with every major CI/CD tool. Trigger Espresso runs on every build and stream device videos and JUnit reports back into your build dashboards. ![Jenkins](https://testingbot.com/assets/integrations/jenkins-e586a1c4af833e4c8d38d7b392a4e84bd193f30cae9cf8c9f198a58ec943a1ff.webp) ### [Jenkins](https://testingbot.com/support/integrations/ci-cd/jenkins) ![Bamboo](https://testingbot.com/assets/integrations/bamboo-c8fe09a3cb0b74d6055e3aa6607d77720bfe29ff487b8e3cf825138adf634346.svg) ### [Bamboo](https://testingbot.com/support/integrations/ci-cd/bamboo) ![TeamCity](https://testingbot.com/assets/integrations/teamcity-7e75cdb835903a4983ad276c41d95e569ceac5be486e96ec1f955029ce5bb319.svg) ### [TeamCity](https://testingbot.com/support/integrations/ci-cd/teamcity) ![CircleCI](https://testingbot.com/assets/integrations/circleci-ede029654ff39b60342e4a5bba8fc9a3a49305cc38bda215b7cf8d07e85e0543.webp) ### [CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci) ![GitLab CI](https://testingbot.com/assets/integrations/gitlab-e4c0b6db07d07bc0a400f5606e258f4dc8a51b778e97bbfa51d1d531f3cb049a.webp) ### [GitLab CI](https://testingbot.com/support/integrations/ci-cd/gitlab) ![GitHub](https://testingbot.com/assets/integrations/github-actions-8436242ad9e15381f84f1a98a10dddce15611087f121d6305782e211942b7842.webp) ### [GitHub](https://testingbot.com/support/integrations/ci-cd/github-actions) Spoof device GPS 20+ regions United States United Kingdom Germany France Spain Belgium Australia China Japan Geolocation testing ## Test your Android app from 20+ regions Spoof the device GPS to anywhere in the world and verify **localisation** , **geo-targeting** and **Play Store store-front rules** from any country your customers live in. Secure devices ## Real devices. Wiped after every session. Each Espresso session locks one real Android device for your test only. The moment your test ends, the device is wiped, all installed apps are deleted, caches are cleared, and the device is returned to the pool ready for the next customer. GDPR-compliant, with SAML SSO and audit logs available for enterprise teams. - One real device per session, never shared - Wiped between every session - Encrypted .apk uploads with per-app retention controls [Visit TestingBot Trust Center](https://trust.testingbot.com/) 1. 1 Your Espresso run locks one real device A pristine Galaxy, Pixel, OnePlus or Xiaomi is reserved for your session only. 2. 2 Tests run natively on real Android hardware Real sensors, real fingerprint reader, real OEM customisations. 3. 3 Device wiped on release Apps removed, caches cleared, accounts wiped. Returned clean to the pool. Secure Tunnel ## Test apps that talk to internal backends Use our [secure tunnel](https://testingbot.com/support/tunnel) so Espresso runs can reach private APIs, staging environments and services running on localhost. Encrypted end-to-end, no inbound firewall rules required, ready out of the box for popular CI providers. ![TestingBot secure tunnel for Espresso](https://testingbot.com/assets/features/tunnel-diagram-poster-8c25e19e6fe4f7276a46a96b31a0ad0e07bb4f5bf98dc192d2b70897cfbb1a26.png) [Read more](https://testingbot.com/support/tunnel) FAQ ## Espresso on TestingBot — frequently asked questions What is Espresso and why run it on TestingBot? Espresso is Google's native UI testing framework for Android. TestingBot runs your Espresso builds on a fleet of real Galaxy, Pixel, OnePlus and Xiaomi devices, so you do not have to maintain devices, OEM updates and ADB tunnels in-house. Upload your `.apk` and `.apk` test bundle through the TestingBot CLI and trigger runs in seconds. Which Android versions are supported? TestingBot supports real Android devices from Android 4.4 through Android 15, refreshed every Google and OEM release. We carry the latest flagship Galaxy, Pixel and OnePlus phones plus a deep legacy fleet for backwards-compat coverage. Can I run on real devices and emulators? Yes. Plans on TestingBot include both real Android devices and Android emulators with Google Play APIs. Toggle per-test with the --real-device flag in the TestingBot CLI. Can I run Espresso in parallel? Yes. Plans on TestingBot include up to 100 parallel Espresso sessions on real devices and emulators. Each session locks a dedicated device, so tests do not interfere with one another. Can I filter which tests run? Yes — use the [test filters API](https://testingbot.com/support/app-automate/espresso/filter) to run a subset of your Espresso suite without rebuilding the bundle. How do I see results? Live in the TestingBot dashboard with HD video, logcat output and screenshots. Programmatically via the [REST API](https://testingbot.com/support/api). Plus a JUnit XML report compatible with every major CI parser. Can Espresso reach my staging backend? Install our [secure tunnel](https://testingbot.com/support/tunnel) and your Espresso runs can reach staging APIs, internal services and localhost without any inbound firewall rules. Is the device cloud GDPR / SOC 2 compliant? TestingBot is not SOC 2 or ISO 27001 certified. We are GDPR-compliant with EU-only data residency, registered at CSA STAR Level 1 (a published self-assessment), and our control set is modelled on ISO 27001 Annex A. Our datacenter operators hold ISO 27001. Devices are wiped between sessions, app uploads are encrypted, and SAML SSO and audit logs are available on enterprise plans. ## Ready to scale your Espresso runs? Read our [Espresso documentation](https://testingbot.com/support/app-automate/espresso) and run your first test on a real Android device in under 5 minutes. [Start a free trial](https://testingbot.com/users/sign_up) Resources you may like [![Android Espresso Tutorial](https://testingbot.com/assets/resources/articles/19-fe4971eb8049097c25f2cc2fce95b71b94f3bb342e094c219b2404c09374f015.webp)](https://testingbot.com/resources/articles/android-espresso-testing) ### [Android Espresso Tutorial](https://testingbot.com/resources/articles/android-espresso-testing) Why Espresso suits Android UI testing, how to add it to a project and write a first test, and how it compares with the alternatives. [Read more: Android Espresso Tutorial](https://testingbot.com/resources/articles/android-espresso-testing) --- URL: https://testingbot.com/features/automation/maestro Maestro Cloud alternative, drop-in CLI # Maestro Cloud Testing on Real iOS and Android Devices TestingBot Maestro Cloud runs your Maestro flow files on real iOS and Android devices, simulators and emulators. Every test executes 2x faster than Maestro Cloud at a third of the cost, with a single CLI swap. [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) 30 days free for Maestro users - **Real devices & sims:** 200+ - **Faster than Maestro Cloud:** 2x - **Cost vs Maestro Cloud:** 1/3 https://www.youtube.com/embed/MMWCkBgXFOU Trusted by some of the world's most innovative companies Drop-in CLI swap ## A drop-in replacement for Maestro Cloud Switch from Maestro Cloud CLI to TestingBot in seconds. Replace `maestro cloud` with `testingbot maestro`. Same flows, same YAML, same arguments. Maestro Cloud `maestro cloud android.apk flows/` TestingBot `testingbot maestro android.apk flows/` ### Quick start ```bash $ npm install -g testingbot-cli $ testingbot maestro app.apk ./flows \ --device "Pixel 8" \ --deviceVersion "14" ``` ### Advanced options ```bash $ testingbot maestro app.apk ./flows \ -e API_URL=https://staging.example.com \ --include-tags "smoke,critical" \ --real-device ``` ### CLI options | Option | Description | | --- | --- | | --device | Device name (e.g. "Pixel 8", "iPhone 15") | | --real-device | Target a physical device (iOS or Android) | | --deviceVersion | OS version (e.g. "14" for Android, "17.0" for iOS) | | --maestro-version | Pin a Maestro version (e.g. "2.0.10") | | -e, --env | Pass environment variables to your tests | | --include-tags | Run only flows with the given tags | | --exclude-tags | Skip flows with the given tags | | --name | Custom name for your test run | | --locale | Set the device locale (e.g. "de\_DE", "fr\_FR") | [View all CLI options](https://testingbot.com/support/app-automate/maestro/options) [Get started free](https://testingbot.com/users/sign_up) How it works ## From local CLI to cloud results in three steps 1 ### Upload your mobile app Push your .apk, .aab or .ipa via the TestingBot CLI or REST API. Builds are stored in your account for fast re-runs. 2 ### Point at your Maestro flows Reuse the same Maestro flow YAML you run locally. No rewrites, no plugins. 3 ### Watch results stream in Flows run in parallel across real devices and surface in your TestingBot dashboard with live video, logs and step-by-step screenshots. Side-by-side ## TestingBot vs Maestro Cloud vs DeviceCloud A side-by-side look at the leading Maestro cloud providers. TestingBot is the cheaper, faster real device cloud with iPhones, iPads, Galaxy and Pixel devices on demand. | Feature | TestingBot ✨ Best value | Maestro Cloud | DeviceCloud | | --- | --- | --- | --- | | 💰 Pricing | 1/3 the cost | 3x more expensive | Pay per test run | | 📱 Real physical devices (iOS and Android) | | | | | 📲 iPhone & iPad simulators | | | Pay extra for specific iOS versions | | 🤖 Android emulators with Google Play APIs | Full support | Limited | Pay extra | | 💳 In-app purchase testing | Available on Android | | | | 🔄 Portrait & landscape mode | Full control | Basic | | | 🌍 Geolocation testing | Any location | | | | 🌐 Device locale testing | 20+ locales | Limited | 9 locales | | 📹 Real-time video streaming | Live streaming | Post-test only | Live streaming | | 📊 Real-time logs | Live streaming | Basic | Live streaming | | ⚡ Performance | 2x faster (Mac Silicon + ARM) | Standard speed | Standard speed | [Start testing with TestingBot](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) or [view pricing](https://testingbot.com/pricing) Deep dive: [Maestro Cloud alternative](https://testingbot.com/maestro-cloud-alternative) · [devicecloud.dev alternative](https://testingbot.com/devicecloud-alternative) Massive parallelism ## Cut Maestro suite time from hours to minutes Run hundreds of Maestro flows in parallel across real devices, simulators and emulators. Same flows, a fraction of the wall-clock time. - Up to 100 concurrent Maestro sessions - Sessions start in seconds, no device queue - Auto-retry, and your own flow sharding works unchanged [Start free](https://testingbot.com/users/sign_up) ![Cut Maestro suite time from hours to minutes](https://testingbot.com/assets/features/selenium/parallel-0229d698e59a8a6b92fe50344a08634624f018331ecf8a849ef83cfb35e635a5.png) Real-time insights ## Watch flows execute in real time Stream live video, see steps as they run, and pull logs from the TestingBot dashboard or via the REST API. No waiting for tests to finish. [Start free](https://testingbot.com/users/sign_up) ![Watch flows execute in real time](https://testingbot.com/assets/support/maestro/results-78a1591a85866ec01fa9a661652d9fd4cf0b24989743d73756e8a991157dd7be.png) Spoof device GPS 20+ regions United States United Kingdom Germany France Spain Belgium Australia China Japan Geolocation testing ## Test your app from 20+ regions Spoof the device GPS to anywhere in the world and verify **localisation** , **geo-targeting** and **store-front rules** from any country your customers live in. Real device cloud ## A real device cloud for Maestro flows TestingBot is a real device cloud built for Maestro. Test on real iPhones, iPads, Galaxy and Pixel devices, plus iOS Simulators and Android Emulators with Google Play APIs. iOS 16 to 26, Android 10 to 16. Latest flagships on day one. [View all devices](https://testingbot.com/support/app-automate/devices) ![Test online on Pixel 8](https://testingbot.com/assets/devices/160h/46-d0ca4c1d1985aefc27c52ba2a1c88748875c574d5358e53adcbdeaee4ed078bb.webp) Pixel 8 ![Test online on Pixel 11](https://testingbot.com/assets/devices/160h/45-b90994534fadde53be99a94521c6ea23a39fdf32ad277240be4e2613540b5019.webp) Pixel 11 ![Test online on iPhone 17 Pro](https://testingbot.com/assets/devices/160h/44-6b4402c6996d9b1620879bb1123c40a6b36abb064e65dca0a1b3f05286779e9c.webp) iPhone 17 Pro ![Test online on iPhone 17 Pro Max](https://testingbot.com/assets/devices/160h/43-59d7e30684332ed4bf25567cbc09ca942eac46d925ccde8352d3e85812480f7d.webp) iPhone 17 Pro Max ![Test online on iPhone 16 Pro](https://testingbot.com/assets/devices/160h/42-de3704079345026847b739a5b1cf5332595904e54ede66abd988a76fb1e6593c.webp) iPhone 16 Pro ![Test online on iPad (9th generation)](https://testingbot.com/assets/devices/160h/41-70f7d18afde7620063a49a76cbfd088f8b59c742f3a5f4dd4a513dd6ec5c676c.webp) iPad (9th generation) Continuous integration ## Plug Maestro into your CI/CD in minutes Trigger Maestro flows on every commit and stream device videos and logs back into Jenkins, GitHub Actions, GitLab CI, CircleCI and more. ![Jenkins](https://testingbot.com/assets/integrations/jenkins-e586a1c4af833e4c8d38d7b392a4e84bd193f30cae9cf8c9f198a58ec943a1ff.webp) ### [Jenkins](https://testingbot.com/support/integrations/ci-cd/jenkins) ![Bamboo](https://testingbot.com/assets/integrations/bamboo-c8fe09a3cb0b74d6055e3aa6607d77720bfe29ff487b8e3cf825138adf634346.svg) ### [Bamboo](https://testingbot.com/support/integrations/ci-cd/bamboo) ![TeamCity](https://testingbot.com/assets/integrations/teamcity-7e75cdb835903a4983ad276c41d95e569ceac5be486e96ec1f955029ce5bb319.svg) ### [TeamCity](https://testingbot.com/support/integrations/ci-cd/teamcity) ![CircleCI](https://testingbot.com/assets/integrations/circleci-ede029654ff39b60342e4a5bba8fc9a3a49305cc38bda215b7cf8d07e85e0543.webp) ### [CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci) ![GitLab CI](https://testingbot.com/assets/integrations/gitlab-e4c0b6db07d07bc0a400f5606e258f4dc8a51b778e97bbfa51d1d531f3cb049a.webp) ### [GitLab CI](https://testingbot.com/support/integrations/ci-cd/gitlab) ![GitHub](https://testingbot.com/assets/integrations/github-actions-8436242ad9e15381f84f1a98a10dddce15611087f121d6305782e211942b7842.webp) ### [GitHub](https://testingbot.com/support/integrations/ci-cd/github-actions) Secure Tunnel ## Test apps that talk to internal backends Use our [secure tunnel](https://testingbot.com/support/tunnel) so Maestro flows can reach private APIs, staging environments and services running on localhost. Encrypted end-to-end, no inbound firewall rules required, ready out of the box for popular CI providers. ![TestingBot secure tunnel for Maestro](https://testingbot.com/assets/features/tunnel-diagram-poster-8c25e19e6fe4f7276a46a96b31a0ad0e07bb4f5bf98dc192d2b70897cfbb1a26.png) [Read more](https://testingbot.com/support/tunnel) FAQ ## Maestro on TestingBot, frequently asked questions What is Maestro and why run it on TestingBot? Maestro is the open-source mobile UI testing framework from mobile.dev, used for native iOS, Android, Flutter and React Native automation testing. TestingBot runs the same Maestro flows you run locally on real iPhones, iPads and Android devices in the cloud. Swap `maestro cloud` for `testingbot maestro` and your existing flows just work. Is TestingBot really a drop-in replacement for Maestro Cloud? Yes. The CLI signature is identical (app + flows + flags), so any script or CI job that calls Maestro Cloud can call TestingBot instead. Same YAML flows, same arguments, same exit codes. Which Maestro versions are supported? TestingBot supports every recent Maestro release, and you can pin a specific Maestro server version per-test using the `--maestro-version` flag. Newer versions are typically available within days of release. Can I run Maestro flows in parallel? Yes. Plans on TestingBot include up to 100 parallel Maestro sessions on real devices, simulators and emulators. Use `--include-tags` / `--exclude-tags` to shard intelligently and the dashboard will roll the results back together. Real devices, simulators or both? All three. TestingBot runs Maestro flows on real iPhones, iPads, Galaxy and Pixel devices, plus iOS Simulators on Apple Silicon and Android Emulators with Google Play APIs. Pick per-flow with the `--real-device` flag. How do I see results? Live in the TestingBot dashboard with video streaming, step-by-step screenshots and Maestro logs. Programmatically via our [REST API](https://testingbot.com/support/api). Or piped into Jenkins, GitHub Actions, GitLab CI and any CI that consumes JSON. Can Maestro flows reach my staging backend? Install our [secure tunnel](https://testingbot.com/support/tunnel) and your Maestro flows can reach staging APIs, internal services and localhost without any inbound firewall rules. Is the device cloud GDPR / SOC 2 compliant? TestingBot is not SOC 2 or ISO 27001 certified. We are GDPR-compliant with EU-only data residency, registered at CSA STAR Level 1 (a published self-assessment), and our control set is modelled on ISO 27001 Annex A. Our datacenter operators hold ISO 27001. Devices are factory-reset between sessions, app uploads are encrypted, and SAML SSO and audit logs are available on enterprise plans. What is Maestro? Maestro is an open-source mobile UI testing framework built by mobile.dev. It lets you write declarative YAML flows that drive native iOS and Android apps, plus Flutter and React Native, with built-in retries, auto-waiting and screenshots. Teams use Maestro for end-to-end testing, smoke tests and regression coverage. TestingBot runs the same Maestro flows on real iPhones, iPads, Galaxy and Pixel devices, simulators and emulators in the cloud, with parallel execution and CI/CD integration. Is Maestro free? Yes, the Maestro CLI and framework are free. Mobile.dev releases Maestro under the Apache 2.0 license, so you can install it, write flows and run them locally at no cost. The cost question for teams is usually how to run Maestro at scale across real devices and parallel workers, which is what TestingBot provides on flat plans at a third of the cost of Maestro Cloud. Is Maestro open source? Maestro is open source. The framework is maintained by mobile.dev on GitHub under the Apache 2.0 license, with a public roadmap and active community. TestingBot complements the open-source framework with a managed device cloud for real iPhones, iPads and Android devices, parallel runs and detailed reporting. Which platforms and frameworks does Maestro support? Maestro supports native iOS apps, native Android apps, Flutter and React Native, and works on real devices, iOS Simulators and Android Emulators. The same YAML flow runs across platforms with minor selector differences. TestingBot exposes all of these targets through the standard Maestro CLI, so you can pick a real iPhone, an iOS Simulator on Apple Silicon, an Android Emulator with Google Play APIs or a real Galaxy or Pixel device per flow. What is the best Maestro Cloud alternative? Maestro Cloud (run by Mobile Dev Inc. at maestro.dev) is the official managed cloud for Maestro flows. TestingBot is the leading independent Maestro Cloud alternative, with an identical CLI, real iPhones, iPads, Galaxy and Pixel devices alongside iOS Simulators and Apple Silicon Android Emulators, roughly a third of the cost per run on flat plans with no per-test surprises, plus parallel execution, geolocation testing in 20+ countries, secure tunnel access for staging environments, real-time video streaming and CI/CD integrations out of the box. How does TestingBot compare to DeviceCloud? DeviceCloud is a Maestro-focused cloud that runs flows on parallel iOS Simulators and Android Emulators only, with no real-device support, on subscription plans where extra runs are billed per test. TestingBot offers a true real device cloud, with real iPhones, iPads, Galaxy and Pixel devices alongside the same iOS Simulators and Android Emulators, on flat plans with predictable pricing, real-time video and logs, geolocation testing in 20+ countries, device locale testing in 20+ locales, and Apple Silicon performance for iOS Simulators. Both expose a Maestro-compatible CLI, so migrating between them is a one-line change. ## More reasons to love TestingBot ### 24/7 support Got questions? Ask in the 24/7 in-app customer [chat support](https://testingbot.com#) or [send us an email](https://testingbot.com/contact/new). We are here to help you! TestingBot Support Hey 👋 How can we help? I need support ### Trusted by over 14,000 teams 6M+ Tests 120k+ Users 400+ Enterprises 106 Countries [Request a demo](https://testingbot.com/demo) ### Developer Documentation Everything you need to integrate TestingBot into your workflow. Browse guides, API references and SDKs for web and mobile testing. - [Maestro Quickstart](https://testingbot.com/support/app-automate/maestro) - [Maestro CLI options](https://testingbot.com/support/app-automate/maestro/options) - [CI/CD integration guides](https://testingbot.com/support/integrations/ci-cd) [Explore Documentation](https://testingbot.com/support) ### Enterprise-grade Security STAR Level 1 Certified GDPR Compliant SAML SSO Single Sign-On 99.9% Uptime SLA [Visit TestingBot Trust Center](https://trust.testingbot.com/) ## Ready to scale your Maestro testing? Read our [Maestro Cloud Testing](https://testingbot.com/support/app-automate/maestro) documentation and run your first flow in under 5 minutes. [Start a free trial](https://testingbot.com/users/sign_up) --- URL: https://testingbot.com/maestro-cloud-alternative TestingBot vs Maestro Cloud # The real-device Maestro Cloud alternative Run your existing Maestro flow files on real iPhones, iPads, Android phones and tablets, plus simulators and emulators. Flat monthly pricing instead of negotiated quotes. EU data residency for procurement. Switching from Maestro Cloud? Maestro users get a **30-day free trial** , no credit card required. [Start your free trial](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) - **Real devices & sims:** 200+ - **EU hosted since:** 2012 - **Frameworks:** 8+ Trusted by some of the world's most innovative companies At a glance ## Why teams pick TestingBot over Maestro Cloud Six concrete differences that show up in the first sprint after switching. - ### Real iPhones and Android devices Run flows on actual hardware with biometrics, push, camera and sensors. Maestro Cloud's public offering is configurable models, not your real device matrix. - ### Every framework, one place Same account runs Maestro, Appium, XCUITest, Espresso, Playwright and Selenium. Maestro Cloud is Maestro-only by design. - ### Published flat pricing Public monthly tiers by concurrent sessions. No per-test meter, no sales-led negotiation. Maestro Cloud pricing is quote-only. - ### EU hosted, GDPR ready Test data, video and logs stay in Europe (EU and Switzerland). GDPR, STAR L1, ISO 27001 datacenters and SAML SSO included. Maestro Cloud is US-hosted. - ### Tunnel to internal backends TestingBot Tunnel exposes staging and pre-prod backends to the cloud devices over a secure connection. Maestro Cloud has no documented equivalent. - ### Drop-in CLI swap Same flow files, same YAML, same arguments. Replace maestro cloud with testingbot maestro and your suite runs on TestingBot in minutes. Compare ## TestingBot vs Maestro Cloud, feature by feature Capabilities side by side, based on publicly documented Maestro Cloud features as of October 2026. TestingBot vs Maestro Cloud, feature comparison.| Capability | Real devices TestingBot | Maestro Cloud | | --- | --- | --- | | Starting price | Published tiers See [pricing](https://testingbot.com/pricing) | Not published Sales-led quote | | Real iPhones and iPads | | Not publicly offered | | Real Android phones and tablets | | Limited | | iOS simulators and Android emulators | | | | Appium, XCUITest, Espresso | | – | | Playwright and Selenium for web | | – | | Local tunnel to internal backends | TestingBot Tunnel | – | | Biometrics, camera, push, sensors | Real device support | Not on simulators | | CI/CD integrations | GitHub, GitLab, Jenkins, CircleCI, Bitbucket | GitHub Actions, Bitrise | | Pricing transparency | Published monthly tiers | Sales-led quote | | Free trial | 30 days, no card | 7 days | | EU data residency | | – | | Compliance | GDPR, STAR L1, ISO 27001 datacenters, SAML SSO | SOC 2 (per public claims) | Pricing ## What Maestro testing costs on each platform List prices as published by each vendor, checked on 19 August 2026. Shown in the currency each vendor publishes, without conversion. ### TestingBot Billed per plan, every framework included. - €50 / month, billed annually, for 1 parallel test and 1,000 automated minutes - €90 / month, billed annually, for **unlimited** automated minutes - Higher parallelism on the same plans: 2, 4, 8, 16 or 24 parallel tests - Appium, XCUITest, Espresso and Selenium included, not priced separately See the [full pricing page](https://testingbot.com/pricing) for month-to-month rates. ### Maestro Cloud Billed per concurrent device. - Free for the Local tier, running on your own devices with the open-source CLI - $250 / device / month for Cloud, priced on maximum concurrent executions, with unlimited hosted runs - Enterprise Cloud is listed as custom pricing with no public figure - Pricing for web and Chromium devices is not published Source: [maestro.dev/pricing](https://maestro.dev/pricing), 19 August 2026. The closest like-for-like comparison is a single concurrent slot with unlimited runs: **€90 per month on TestingBot Automated Pro** against **$250 per device per month on Maestro Cloud**. The two are priced in different currencies and structured differently, so compare the tier that matches your concurrency rather than the headline number. Maestro's Local tier remains free if you are happy running on hardware you own and maintain. Migrate in under an hour ## Switch from Maestro Cloud in four steps Your flow files do not change. The migration is a CLI swap and a credentials update. The [Maestro Cloud migration guide](https://testingbot.com/support/app-automate/maestro/migrate-from-maestro-cloud) maps every `maestro cloud` flag to its TestingBot equivalent. 1. 1 ### Create a free account Sign up at testingbot.com and grab your TB\_KEY and TB\_SECRET from the account dashboard. 2. 2 ### Install the CLI `npm install -g @testingbot/cli`. Works alongside your existing Maestro CLI install. 3. 3 ### Swap the run command Replace `maestro cloud` with `testingbot maestro`. Add `--real-device --device "iPhone 15"` to target physical hardware. 4. 4 ### Run your suite Trigger a smoke test, watch the live video, fetch results via the [REST API](https://testingbot.com/support/app-automate/maestro/api) or dashboard. Coverage ## Real iPhones, Pixels and Galaxys ready for your flows A grid of physical iPhones, iPads, Android phones and tablets sits behind every `testingbot maestro --real-device` command. Pin a specific model or use a regex like `".*Pixel.*"` to allocate any available match. Simulators and emulators are included on the same account for fast pre-merge smoke runs. Drop `--real-device` and the same flow runs in seconds on a clean simulator. [See supported devices](https://testingbot.com/support/app-automate/devices) - **Real iOS devices:** iPhone 6 → 17 Pro Max - **Real Android devices:** Pixel, Galaxy, Huawei - **iOS simulators:** iOS 12 → 26 - **Android emulators:** API 24 → 36 Frameworks ## Maestro is one of many on TestingBot Engineering managers consolidate Maestro, Appium, XCUITest, Espresso, Playwright and Selenium in one place. Maestro Cloud is Maestro-only by design. [![Maestro Testing](https://testingbot.com/assets/integrations/maestro-9d782f30f13fb4efaf9928b704c7a787eb14cb9e5b20c650137bb65892334b8b.svg)](https://testingbot.com/features/automation/maestro) ### [Maestro](https://testingbot.com/features/automation/maestro) YAML-driven mobile UI testing. Run flows on real devices, simulators and emulators. [![Appium Testing](https://testingbot.com/assets/support/appium-4b35a8bb56bae236bd84c539cee9b8bbb489a55ddf48e32d059c86685d230354.svg)](https://testingbot.com/features/automation/appium) ### [Appium](https://testingbot.com/features/automation/appium) Mobile automation on real iOS and Android devices for deeper white-box scenarios. [![XCUITest Testing](https://testingbot.com/assets/integrations/xcuitest-4a935f2368661d89d3559b3c5af3377236029478ae76bb43253f3c3bee712096.svg)](https://testingbot.com/features/automation/xcuitest) ### [XCUITest](https://testingbot.com/features/automation/xcuitest) Apple's native iOS UI testing framework. Runs against real iPhones and iPads. [![Espresso Testing](https://testingbot.com/assets/integrations/espresso-a8ff30da0c385f89f4e694bb29fc4779ad132e8e3f4a777a38e608e0492ff610.svg)](https://testingbot.com/features/automation/espresso) ### [Espresso](https://testingbot.com/features/automation/espresso) Google's instrumentation framework for native Android apps. Same device fleet. [![Playwright Testing](https://testingbot.com/assets/support/playwright-2dcb4e75b55ac0d1135cfa47be6137b13f46a2047ce78b96a34f51b0e2cc0d96.svg)](https://testingbot.com/features/automation/playwright) ### [Playwright](https://testingbot.com/features/automation/playwright) Cross-browser automation with auto-waits, network interception and trace viewer. [![Selenium Testing](https://testingbot.com/assets/support/selenium-266ecc0599068fc6eca0bcd9174fb46053c95cebe1c66f7a50c480f3f1a52929.svg)](https://testingbot.com/features/automation/selenium) ### [Selenium](https://testingbot.com/features/automation/selenium) Industry-standard browser automation. Java, Python, Ruby, JavaScript, C# and more. [![Cypress Testing](https://testingbot.com/assets/support/cypress-409c9fd3f79115ed7676aa1df4f6a217dc4402b3366a61984924aec7b4fefa97.svg)](https://testingbot.com/features/automation/cypress) ### [Cypress](https://testingbot.com/features/automation/cypress) Modern JavaScript end-to-end testing on Chrome, Firefox and Edge in parallel. [![Puppeteer Testing](https://testingbot.com/assets/support/puppeteer-0a8cdd38af69dabd46638afbb3bffc8a372902949aa599663ec379f3e875263c.svg)](https://testingbot.com/features/automation/puppeteer) ### [Puppeteer](https://testingbot.com/features/automation/puppeteer) Headless Chrome and Edge automation, ideal for fast scripted browser tasks. Speed ## Parallel Maestro runs by concurrent sessions Cut total Maestro suite duration by sharding flows across concurrent sessions. Each shard runs on its own device or simulator, fully isolated. The TestingBot CLI handles `--shard-split` and `--shard-all` identically to the Maestro CLI. Predictable flat tiers, not per-test credits. Add concurrency by upgrading the plan, not by buying ad-hoc credit packs. [See pricing](https://testingbot.com/pricing) parallel-shards.sh ``` $ testingbot maestro app.apk ./flows \ --real-device \ --device "Pixel 8" \ --shard-split 4 shard 1 of 4 → Pixel 8 ✓ 12 flows · 3m 24s shard 2 of 4 → Pixel 8 ✓ 12 flows · 3m 18s shard 3 of 4 → Pixel 8 ✓ 12 flows · 3m 31s shard 4 of 4 → Pixel 8 ✓ 12 flows · 3m 20s Total: 48 flows · 3m 31s wall clock ``` Why switch ## Why teams move from Maestro Cloud to TestingBot ### They need real hardware Payment, KYC, biometrics, push, camera. Simulators report synthetic results. Real iPhones and Android devices surface the bugs that ship to App Store and Play Store users. ### They need more than Maestro Legacy Appium and XCUITest suites still exist. Running them on the same vendor as Maestro flows simplifies billing, security review and SSO. ### Procurement asks for EU GDPR, CSA STAR and ISO 27001 datacenters are table stakes for European fintech, health and govtech buyers. TestingBot has been hosted in Europe since 2012. ### Pricing predictability Flat monthly tiers that scale by concurrency, not per-test credits. Finance can budget. Engineering can scale without raising a ticket. ### Tunnel to internal backends Maestro suites that need to hit staging or pre-prod backends behind the firewall use TestingBot Tunnel without exposing services publicly. ### Documentation that ships Working YAML examples for GitHub Actions, GitLab CI, Jenkins, CircleCI and Bitbucket Pipelines, plus a full REST API reference and a step-by-step Maestro guide. FAQ ## Frequently asked questions What is the difference between TestingBot and Maestro Cloud? Maestro Cloud is the cloud service operated by mobile-dev-inc, the maintainers of the Maestro framework. It executes Maestro flows in their own infrastructure with configurable device models. TestingBot is an independent cloud test platform that runs Maestro flows on real iPhones and Android devices, simulators and emulators, alongside Appium, XCUITest, Espresso and Playwright, on a flat monthly plan with EU data residency. Can I keep my Maestro .flow files when moving to TestingBot? Yes. TestingBot supports the standard Maestro YAML flow syntax. Your existing .flow files run unchanged. The migration is a CLI swap: replace maestro cloud with testingbot maestro and your suite executes on TestingBot's grid. Does TestingBot support real iPhones and iPads for Maestro? Yes. TestingBot provides physical iPhones, iPads, Android phones and tablets, plus iOS simulators and Android emulators, all from the same CLI. Use --real-device to target physical hardware, or omit the flag to run on a simulator or emulator. How does pricing compare to Maestro Cloud? TestingBot publishes flat monthly plans based on the number of concurrent sessions you need. There is no per-test charge and no execution-minute meter. Maestro Cloud pricing is not published publicly; teams typically negotiate per workspace. The TestingBot pricing page lists the current concurrent-session tiers. What about Appium, XCUITest and other frameworks? On TestingBot, the same account covers Maestro plus Appium, XCUITest, Espresso, Playwright and Selenium. Engineering managers running mixed mobile and web suites can consolidate on one vendor instead of one per framework. Is TestingBot hosted in the EU? Is it GDPR compliant? Yes. TestingBot is operated from Europe (EU and Switzerland) and has been hosted in Europe since 2012. We are GDPR-compliant, registered in the Cloud Security Alliance STAR Level 1 program (self-assessment), operate from ISO 27001 certified datacenters, and support SAML SSO on team plans. How long does it take to migrate from Maestro Cloud? Most teams migrate in under an hour. The Maestro .flow files do not change. Replace the CLI command, swap two environment variables, and run. Does TestingBot offer a free trial for Maestro users? Yes. TestingBot offers a 30-day free trial for teams coming from Maestro Cloud, no credit card required. Run your existing flow files immediately on real iPhones, Pixels, simulators and emulators. Can I run Maestro tests from GitHub Actions on TestingBot? Yes. TestingBot has dedicated CI/CD integration docs with working YAML examples for GitHub Actions, GitLab CI, Jenkins, CircleCI and Bitbucket Pipelines. Parallel sharding works out of the box. Does TestingBot support tunneling to internal staging backends? Yes. TestingBot Tunnel exposes your local or staging backends to the cloud devices over a secure connection. Maestro flows can hit internal services without exposing them publicly. ## Run Maestro on real devices for 30 days No credit card. Your flow files run unchanged. Real iPhones, Android devices, simulators and emulators. [Start free trial](https://testingbot.com/users/sign_up) --- URL: https://testingbot.com/devicecloud-alternative TestingBot vs devicecloud.dev # The real-device devicecloud.dev alternative devicecloud.dev runs Maestro on iOS simulators and Android emulators. TestingBot runs Maestro on the same simulators and emulators, plus real iPhones, Pixels and Galaxys with biometrics, push, camera and sensors. Flat monthly pricing instead of per-test credits. EU data residency. **30-day free trial** , no credit card required. [Start your free trial](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) - **Real iOS & Android:** 200+ - **EU hosted since:** 2012 - **Pricing model:** Flat Trusted by some of the world's most innovative companies At a glance ## Six reasons to pick TestingBot over devicecloud.dev Each one shows up in the first sprint after switching. - ### Real iPhones and Pixels Real biometrics, real APNs and FCM push, real camera and sensors. devicecloud.dev cannot reproduce any of these because its cloud is simulators and emulators only. - ### Flat monthly pricing Published tiers by concurrent sessions. No per-test credit meter, no surprise top-ups when a long-running suite eats the credit pool. - ### Tunnel to internal backends TestingBot Tunnel exposes staging and pre-prod backends to the cloud devices over a secure connection. devicecloud.dev has no documented equivalent. - ### EU hosted, GDPR ready Test data, video and logs stay in Europe (EU and Switzerland). GDPR, STAR L1, ISO 27001 datacenters and SAML SSO included. devicecloud.dev is operated from the UK and US. - ### Every framework, one place Maestro alongside Appium, XCUITest, Espresso, Playwright and Selenium. devicecloud.dev is Maestro-only. - ### Drop-in CLI swap Replace dcd cloud with testingbot maestro. Same flow files, same YAML, same arguments. Migration is a single shell line. Compare ## TestingBot vs devicecloud.dev, feature by feature Capabilities and pricing side by side, as of October 2026. TestingBot vs devicecloud.dev, feature and pricing comparison.| Capability | Real devices TestingBot | devicecloud.dev | | --- | --- | --- | | Entry price | Published tiers See [pricing](https://testingbot.com/pricing) | $25 / month Pro plan + $25 credit | | Max plan | Flat by concurrency | $200 / month 20 + 20 parallel, sims only | | Real iPhones and iPads | | – | | Real Android phones and tablets | | – | | iOS simulators and Android emulators | | | | Biometrics, camera, push, sensors | Real device support | – | | Pricing model | Flat monthly | Monthly + credits | | Local tunnel to internal backends | TestingBot Tunnel | – | | Multi-framework support | Appium, XCUITest, Espresso, Playwright, Selenium | Maestro only | | EU data residency | | – | | Compliance | GDPR, STAR L1, ISO 27001 datacenters, SAML SSO | Not published | | CI/CD integrations | GitHub, GitLab, Jenkins, CircleCI, Bitbucket | GitHub Actions | | Free trial | 30 days, no card | $20 starter credit | Why real devices ## What emulators and simulators cannot verify Maestro is the same framework on both clouds. What changes is the surface the .flow file runs against. Emulators and simulators cover most layout, navigation and form bugs, but they cannot reproduce these failure modes: - ● **Biometrics:** Face ID, Touch ID, Android BiometricPrompt. Simulators can fake a result, but they cannot reproduce the OS-level interstitial, the timing or the fallback paths. - ● **Push notifications:** Real APNs and FCM delivery, foreground vs background handling, deep links from a real payload. Simulators cannot register with the real push services. - ● **Camera, microphone, sensors:** QR scanning, document capture, gyroscope, accelerometer, real GPS. Emulators report fixed or synthetic values. - ● **Real network behaviour:** Carrier handover, captive portals, IPv6, real WebView versions across OEMs. Emulators run a vanilla AOSP WebView. - ● **OEM Android skins:** Samsung One UI, Xiaomi HyperOS, Pixel stock all render and gesture differently. Emulators ship vanilla AOSP only. Payment flows, onboarding with KYC, login with biometrics and any feature behind a push notification typically fail or pass differently on real hardware. Running a Maestro suite that includes those flows on emulators only means those tests are not actually being exercised. Migrate in under an hour ## Switch from devicecloud.dev in four steps Your flow files do not change. The migration is a CLI swap and a credentials update. 1. 1 ### Create a free account Sign up at testingbot.com and grab your TB\_KEY and TB\_SECRET from the account dashboard. 2. 2 ### Install the CLI `npm install -g @testingbot/cli`. Works alongside your existing Maestro and dcd CLI installs. 3. 3 ### Swap the run command Replace `dcd cloud` with `testingbot maestro`. Add `--real-device --device "Pixel 8"` for physical hardware. 4. 4 ### Run your suite Trigger a smoke test, watch the live video, fetch results via the [REST API](https://testingbot.com/support/app-automate/maestro/api) or dashboard. Coverage ## 200+ real iPhones, Pixels and Galaxys A grid of physical iPhones, iPads, Android phones and tablets sits behind every `testingbot maestro --real-device` command. Pin a specific model or use a regex like `".*Galaxy.*"` to allocate any available matching device. Simulators and emulators are included on the same account for fast pre-merge smoke runs. Drop `--real-device` and the same flow runs in seconds on a clean simulator, just like on devicecloud.dev. [See supported devices](https://testingbot.com/support/app-automate/devices) - **Real iOS devices:** iPhone 6 → 15 Pro - **Real Android devices:** Pixel, Galaxy, Xiaomi - **iOS simulators:** iOS 12 → 17 - **Android emulators:** API 24 → 34 parallel-shards.sh ``` $ testingbot maestro app.apk ./flows \ --real-device \ --device "Pixel 8" \ --shard-split 4 shard 1 of 4 → Pixel 8 ✓ 12 flows · 3m 24s shard 2 of 4 → Pixel 8 ✓ 12 flows · 3m 18s shard 3 of 4 → Pixel 8 ✓ 12 flows · 3m 31s shard 4 of 4 → Pixel 8 ✓ 12 flows · 3m 20s Total: 48 flows · 3m 31s wall clock ``` Speed ## Parallel Maestro on real devices, not just emulators Shard flows with `--shard-split` or `--shard-all` across concurrent real devices, simulators or emulators. The TestingBot CLI uses the same shard semantics as the Maestro CLI. Add concurrency by upgrading the plan, not by buying ad-hoc credit packs. devicecloud.dev's per-test credit model penalises long suites and short flows alike. [See pricing](https://testingbot.com/pricing) Why switch ## Why teams move from devicecloud.dev to TestingBot ### They need real hardware Payment, KYC, biometrics, push, camera. Simulators report synthetic results. Real iPhones and Pixels surface the bugs that ship to App Store and Play Store users. ### Predictable pricing Flat monthly tiers that scale by concurrency, not per-test credits. Finance can budget. Engineering can scale without raising a top-up ticket. ### Procurement asks for EU GDPR, CSA STAR and ISO 27001 datacenters are table stakes for European fintech, health and govtech buyers. devicecloud.dev is operated from the UK and US. ### More than just Maestro Legacy Appium and XCUITest suites coexist with new Maestro flows. One vendor, one billing relationship, one SSO. ### Tunnel to internal backends Maestro suites that hit staging or pre-prod backends behind the firewall use TestingBot Tunnel without exposing services publicly. ### Mature CI/CD Working YAML examples for GitHub Actions, GitLab CI, Jenkins, CircleCI and Bitbucket Pipelines, plus a documented REST API. FAQ ## Frequently asked questions What is the difference between TestingBot and devicecloud.dev? TestingBot runs Maestro flows on real iPhones, iPads, Android phones and tablets, plus iOS simulators and Android emulators, in a managed EU and US cloud, with flat monthly pricing. devicecloud.dev runs Maestro flows on iOS simulators and Android emulators only, and charges per test credit on top of a monthly plan. Does devicecloud.dev support real devices? No. devicecloud.dev's cloud is iOS simulators and Android emulators only. TestingBot supports both simulators/emulators and a fleet of real physical iPhones, iPads and Android phones. Can I keep my Maestro flow files as they are? Yes. Your existing Maestro YAML .flow files run unchanged on TestingBot. Replace dcd cloud or maestro cloud with the testingbot maestro CLI command and your existing flows execute against TestingBot's grid. How does pricing compare? TestingBot publishes flat monthly plans based on concurrent sessions. There is no per-test credit metering. devicecloud.dev's Pro plan starts at $25/month with $25 of credit and 5 parallel iOS plus 5 Android. The Max plan is $200/month with 20 parallel iOS plus 20 Android. Heavy users on devicecloud.dev typically burn through credits and add packs. Is TestingBot hosted in the EU? Yes. TestingBot has been hosted in Europe since 2012 and is GDPR-compliant, registered in the Cloud Security Alliance STAR Level 1 program (self-assessment), operates from ISO 27001 certified datacenters, and supports SAML SSO. devicecloud.dev runs from the UK and US. How do I migrate from devicecloud.dev to TestingBot? Install the TestingBot CLI (npm install -g @testingbot/cli), set your TB\_KEY and TB\_SECRET environment variables, then replace dcd cloud with testingbot maestro in your run script. No flow file changes required. What about CI/CD? TestingBot has dedicated CI/CD integration docs with examples for GitHub Actions, GitLab CI, Jenkins, CircleCI and Bitbucket Pipelines. Parallel sharding works out of the box on real devices, simulators and emulators. Does TestingBot offer a free trial? Yes. Maestro users get a 30-day free trial on TestingBot, no credit card required. Run your existing flow files immediately on real iPhones, Pixels, simulators and emulators. Why does running on real devices matter for Maestro? Emulators and simulators cannot reproduce biometrics (Face ID, Touch ID, BiometricPrompt), real APNs and FCM push delivery, camera, microphone, real sensor data, OEM Android skins (One UI, HyperOS), or real carrier and network behaviour. Payment flows, KYC and biometric login frequently pass on emulators and fail on real hardware. ## Try Maestro on real devices for 30 days No credit card. Your flow files run unchanged. Real iPhones, Pixels, simulators and emulators in one parallel suite. [Start free trial](https://testingbot.com/users/sign_up) --- URL: https://testingbot.com/features/automation/xcuitest XCUITest cloud # Run XCUITest on real iPhones & iPads Run your XCUITest builds on real iPhones and iPads from iOS 12 through iOS 27 with a single CLI command. Massive parallelism, video & log artifacts and a 99.9% uptime SLA. Choosing a framework? See [Appium vs XCUITest](https://testingbot.com/appium-vs-xcuitest), or the [full automation framework lineup](https://testingbot.com/features). [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) - **Real Apple devices:** 100+ - **iOS versions:** 12 → 27 - **Uptime SLA:** 99.9% Trusted by some of the world's most innovative companies 30-second answer ## Why run XCUITest on TestingBot? Real iPhones and iPads Run on a fleet of real Apple devices, not the iOS Simulator. Latest iPhone the day Apple ships it, plus a deep legacy fleet for backwards-compat coverage. Replace your in-house Apple lab Skip the cabinet of dusty iPhones, the swollen batteries and the MDM headache. TestingBot has run iOS tests on real devices for 5+ years. Native Swift & Objective-C Use the same XCUITest framework Apple ships with Xcode. Same APIs, same XCTest assertions, real native performance. What is XCUITest? ## XCUITest in 60 seconds ![XCUITest](https://testingbot.com/assets/environments/svg/ios-383468addf160fa18d0e431f529420739d7f7d1206175f682fead627d2e99a52.svg) ### XCUITest Apple · built into Xcode · Swift / Objective-C XCUITest is Apple's official UI testing framework for iOS, iPadOS and tvOS. It is built into Xcode, runs natively on the device through XCTest, and lets you script taps, swipes, scrolls, predicates and accessibility queries against your app's real UI. Because XCUITest runs on the device itself, tests have full access to native gestures, accessibility identifiers and Apple's official element hierarchy. There is no cross-process bridge, XCUITest is the most accurate way to drive a real iPhone. - Native Apple framework, built into Xcode - Swift and Objective-C bindings - Real iPhones, iPads and tvOS. No simulator drift - Predicates, accessibility identifiers, native gestures - XCTest assertions, JUnit XML output for CI One-line CLI ## Run XCUITest on real devices with one command Install the TestingBot CLI, point it at your .ipa and your test bundle, pick a device and you're done. Uploads, capabilities and result polling are all handled for you. install the CLI step 1 ```bash # Install the TestingBot CLI $ npm install -g testingbot-cli # Authenticate (once) — set your KEY and SECRET $ export TESTINGBOT_KEY=your-key $ export TESTINGBOT_SECRET=your-secret # Build your XCUITest bundle with Xcode (one-off) $ xcodebuild build-for-testing \ -scheme MyApp \ -destination 'generic/platform=iOS' ``` run XCUITest step 2 ```bash # Run XCUITest on a real iPhone — one command $ testingbot xcuitest MyApp.ipa MyAppUITests-Runner.zip \ --device "iPhone 16" \ --version "27" \ --real-device # → live results in your TestingBot dashboard # → JUnit XML, video and logs via REST API ``` See the full [XCUITest on TestingBot documentation](https://testingbot.com/support/app-automate/xcuitest) for advanced flags, parallel runs and CI integration. Buy vs build ## TestingBot vs an in-house Apple device lab Maintaining your own iPhones and iPads is expensive. Devices break, batteries swell, iOS updates ship every few weeks. Here is how a hosted Apple device cloud compares. | Capability | TestingBot | In-house Apple lab | | --- | --- | --- | | Device coverage | Real iPhones and iPads from iOS 12 to 27 | Limited to what you can buy and rack | | Time to first test | Minutes, upload your .ipa and trigger via API | Weeks of provisioning, MDM, certificates, tethering | | Parallel scaling | Up to 100 concurrent XCUITest sessions | Capped by the number of devices you own | | Video & logs | Video, device logs, screenshots and JUnit XML | Roll your own recording & log aggregation | | Reliability | 99.9% uptime SLA, 24/7 monitoring | You replace dead batteries and broken screens | | Security | Devices wiped between every session | Manual cleanup, leftover state risks | | Support | 24/7 human support and dedicated CSM | Stack Overflow and your own logs | Capabilities ## Everything XCUITest needs in the cloud Beyond running tests, TestingBot exposes the full set of XCUITest capabilities you would set in Xcode, plus cloud-only features like geolocation and network throttling. - ### [Build & upload](https://testingbot.com/support/app-automate/xcuitest/build-testsuite) Compile your .ipa and .xctest with Xcode, then push to TestingBot Storage. - ### [Test reports](https://testingbot.com/support/app-automate/xcuitest/test-reports) Video, device logs, screenshots and JUnit XML for every run. - ### [Screen orientation](https://testingbot.com/support/app-automate/xcuitest/change-screen-orientation) Force portrait or landscape per test, mid-test rotations included. - ### [Network throttling](https://testingbot.com/support/app-automate/xcuitest/simulate-network-conditions) Simulate 3G, 4G, lossy WiFi and offline conditions per session. - ### [Localization & geo](https://testingbot.com/support/app-automate/xcuitest/set-localization-options) Set device locale, language and GPS coordinates for App Store and back-end checks. Real Apple devices ## iPhone and iPad fleet on demand A fleet of real Apple devices spanning iOS 12 through iOS 27, refreshed every release. Latest flagship iPhones and iPads available the day Apple ships them. [View all devices](https://testingbot.com/support/app-automate/devices) ![Test online on iPhone 17 Pro](https://testingbot.com/assets/devices/160h/44-6b4402c6996d9b1620879bb1123c40a6b36abb064e65dca0a1b3f05286779e9c.webp) iPhone 17 Pro ![Test online on iPhone 17 Pro Max](https://testingbot.com/assets/devices/160h/43-59d7e30684332ed4bf25567cbc09ca942eac46d925ccde8352d3e85812480f7d.webp) iPhone 17 Pro Max ![Test online on iPhone 16 Pro](https://testingbot.com/assets/devices/160h/42-de3704079345026847b739a5b1cf5332595904e54ede66abd988a76fb1e6593c.webp) iPhone 16 Pro ![Test online on iPad (9th generation)](https://testingbot.com/assets/devices/160h/41-70f7d18afde7620063a49a76cbfd088f8b59c742f3a5f4dd4a513dd6ec5c676c.webp) iPad (9th generation) ![Test online on iPhone Air](https://testingbot.com/assets/devices/160h/38-c00a59944eb2e410d375e3f5b39780043f72446bb33b0aea6a26a0c45f309ea1.webp) iPhone Air ![Test online on iPhone 17](https://testingbot.com/assets/devices/160h/36-06478e9d9935ed794bd3ec02c85ad3a8eafe1de3cc11debd15351c1bdc99005b.webp) iPhone 17 ![Video, screenshots and JUnit XML for every run](https://testingbot.com/assets/features/test-results-315382c39a67ad52167d2a4fd78570c99aea3e0ec386c17e82265851e93cb7b3.png) Test reports ## Video, screenshots and JUnit XML for every run Every XCUITest run ships with a video screencast, device logs, viewport screenshots and a JUnit XML report compatible with every major CI parser. Mark sessions as passed or failed from your tests, fetch artifacts via our [REST API](https://testingbot.com/support/api), and ship results into Jira, Slack or your CI dashboard. - Full HD video for every test run - Device logs, app logs and crash reports - Viewport screenshots on demand - JUnit XML for Jenkins, GitHub Actions, GitLab CI Continuous integration ## Plug XCUITest into your CI/CD in minutes TestingBot integrates with every major CI/CD tool. Trigger XCUITest runs on every build and stream device videos and JUnit reports back into your build dashboards. ![Jenkins](https://testingbot.com/assets/integrations/jenkins-e586a1c4af833e4c8d38d7b392a4e84bd193f30cae9cf8c9f198a58ec943a1ff.webp) ### [Jenkins](https://testingbot.com/support/integrations/ci-cd/jenkins) ![Bamboo](https://testingbot.com/assets/integrations/bamboo-c8fe09a3cb0b74d6055e3aa6607d77720bfe29ff487b8e3cf825138adf634346.svg) ### [Bamboo](https://testingbot.com/support/integrations/ci-cd/bamboo) ![TeamCity](https://testingbot.com/assets/integrations/teamcity-7e75cdb835903a4983ad276c41d95e569ceac5be486e96ec1f955029ce5bb319.svg) ### [TeamCity](https://testingbot.com/support/integrations/ci-cd/teamcity) ![CircleCI](https://testingbot.com/assets/integrations/circleci-ede029654ff39b60342e4a5bba8fc9a3a49305cc38bda215b7cf8d07e85e0543.webp) ### [CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci) ![GitLab CI](https://testingbot.com/assets/integrations/gitlab-e4c0b6db07d07bc0a400f5606e258f4dc8a51b778e97bbfa51d1d531f3cb049a.webp) ### [GitLab CI](https://testingbot.com/support/integrations/ci-cd/gitlab) ![GitHub](https://testingbot.com/assets/integrations/github-actions-8436242ad9e15381f84f1a98a10dddce15611087f121d6305782e211942b7842.webp) ### [GitHub](https://testingbot.com/support/integrations/ci-cd/github-actions) Spoof device GPS 20+ regions United States United Kingdom Germany France Spain Belgium Australia China Japan Geolocation testing ## Test your iOS app from 20+ regions Spoof the device GPS to anywhere in the world and verify **localisation** , **geo-targeting** and **App Store store-front rules** from any country your customers live in. Secure devices ## Real devices. Wiped after every session. Each XCUITest session locks one real iPhone or iPad for your test only. The moment your test ends, the device is wiped, all installed apps are deleted, caches are cleared, and the device is returned to the pool ready for the next customer. GDPR-compliant, with SAML SSO and audit logs available for enterprise teams. - One real device per session, never shared - Wiped between every session - Encrypted .ipa uploads with per-app retention controls [Visit TestingBot Trust Center](https://trust.testingbot.com/) 1. 1 Your XCUITest run locks one real device A pristine iPhone or iPad is reserved for your session only. 2. 2 Tests run natively on real Apple hardware Real sensors, real accelerometer, real Touch ID and Face ID flows. 3. 3 Device wiped on release Apps removed, caches cleared, accounts wiped. Returned clean to the pool. Secure Tunnel ## Test apps that talk to internal backends Use our [secure tunnel](https://testingbot.com/support/tunnel) so XCUITest runs can reach private APIs, staging environments and services running on localhost. Encrypted end-to-end, no inbound firewall rules required, ready out of the box for popular CI providers. ![TestingBot secure tunnel for XCUITest](https://testingbot.com/assets/features/tunnel-diagram-poster-8c25e19e6fe4f7276a46a96b31a0ad0e07bb4f5bf98dc192d2b70897cfbb1a26.png) [Read more](https://testingbot.com/support/tunnel) FAQ ## XCUITest on TestingBot — frequently asked questions What is XCUITest and why run it on TestingBot? XCUITest is Apple's native UI testing framework for iOS, iPadOS and tvOS. TestingBot runs your XCUITest builds on a fleet of real iPhones and iPads, so you do not have to maintain devices, certificates and MDM in-house. Upload your `.ipa` and `.xctest` through our REST API and trigger runs in seconds. Which iOS versions are supported? TestingBot supports real iOS devices from iOS 12 through iOS 27, refreshed every Apple release. We carry both the latest flagship iPhones and a deep legacy fleet for backwards-compat coverage. Do I need a Mac to use XCUITest on TestingBot? You need Xcode to build your .ipa and .xctest bundle, but the actual test run happens on TestingBot. Many teams build in CI on a Mac runner, then push the artifacts to TestingBot for the device run. Can I run XCUITest in parallel? Yes. Plans on TestingBot include up to 100 parallel XCUITest sessions on real devices. Each session locks a dedicated iPhone or iPad, so tests do not interfere with one another. Do you provide physical devices? Yes, we provide physical iPhones and iPads, not simulators. Each test run locks a real device for your session only, with native performance and access to real sensors, Touch ID and Face ID. After your session ends, the device is wiped and returned clean to the pool. How do I see results? Live in the TestingBot dashboard with HD video, screenshots and device logs. Programmatically via the [REST API](https://testingbot.com/support/api). Plus a JUnit XML report compatible with every major CI parser. Can XCUITest reach my staging backend? Install our [secure tunnel](https://testingbot.com/support/tunnel) and your XCUITest runs can reach staging APIs, internal services and localhost without any inbound firewall rules. Is the device cloud GDPR / SOC 2 compliant? TestingBot is not SOC 2 or ISO 27001 certified. We are GDPR-compliant with EU-only data residency, registered at CSA STAR Level 1 (a published self-assessment), and our control set is modelled on ISO 27001 Annex A. Our datacenter operators hold ISO 27001. Devices are wiped between sessions, app uploads are encrypted, and SAML SSO and audit logs are available on enterprise plans. ## Ready to scale your XCUITest runs? Read our [XCUITest documentation](https://testingbot.com/support/app-automate/xcuitest) and run your first test on a real iPhone in under 5 minutes. [Start a free trial](https://testingbot.com/users/sign_up) Resources you may like [![Automate native iOS Apps with XCUITest](https://testingbot.com/assets/resources/articles/25-bb2ccb531384c461ef792214839e9e8f99a4b08b1293730ceeab67e4a8352563.webp)](https://testingbot.com/resources/articles/automate-native-ios-apps-xcuitest) ### [Automate native iOS Apps with XCUITest](https://testingbot.com/resources/articles/automate-native-ios-apps-xcuitest) How to automate Apple's own iOS apps with XCUITest, with code for driving Settings, Messages and Photos, and how to extend it to other apps. [Read more: Automate native iOS Apps with XCUITest](https://testingbot.com/resources/articles/automate-native-ios-apps-xcuitest) [![Test your website and mobile apps on iOS 18.](https://testingbot.com/assets/blog/2024/ios18.webp)](https://testingbot.com/blog/ios-18-testing) ### [Test your website and mobile apps on iOS 18.](https://testingbot.com/blog/ios-18-testing) Ready to start testing on iOS 18? Get started today with TestingBot: run automated, visual and manual tests on iOS 18. [Read more: Test your website and mobile apps on iOS 18.](https://testingbot.com/blog/ios-18-testing) --- URL: https://testingbot.com/features/automation/appium Appium 1, 2 & 3 ready # Appium Cloud Automation Run your Appium tests on real iOS and Android devices. Native, hybrid and mobile web apps. Massive parallelism, video & log artifacts and a 99.9% uptime SLA. Comparing frameworks? See [Appium vs Espresso](https://testingbot.com/appium-vs-espresso) and [Appium vs XCUITest](https://testingbot.com/appium-vs-xcuitest). [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) - **Real devices:** 200+ - **Appium versions:** 1, 2 & 3 - **Uptime SLA:** 99.9% _Appium tests running on real iOS, Android, Samsung and Pixel devices in the TestingBot cloud_ Trusted by some of the world's most innovative companies 5-line setup ## Run your existing Appium scripts on real devices Point your Appium driver at the TestingBot hub, supply your app and capabilities, and your test runs on a real iPhone or Android phone in seconds. hub.testingbot.com Copy JavaScript Python Java Ruby C# PHP ```javascript // Run an Appium test on a real iPhone in the TestingBot cloud const { remote } = require('webdriverio'); const driver = await remote({ hostname: 'hub.testingbot.com', port: 443, protocol: 'https', capabilities: { 'platformName': 'iOS', 'appium:deviceName': 'iPhone 15', 'appium:platformVersion': '18', 'appium:app': 'tb://YOUR_APP_ID', 'tb:options': { name: 'Checkout flow', realDevice: true } } }); await driver.deleteSession(); ``` ```python # Run an Appium test on a real Android device in the TestingBot cloud from appium import webdriver from appium.options.android import UiAutomator2Options options = UiAutomator2Options() options.platform_version = '14' options.device_name = 'Galaxy S24' options.app = 'tb://YOUR_APP_ID' options.set_capability('tb:options', {'name': 'Checkout flow', realDevice: True}) driver = webdriver.Remote( 'https://hub.testingbot.com/wd/hub', options=options ) driver.quit() ``` ```java // Run an Appium test on a real iPhone in the TestingBot cloud XCUITestOptions options = new XCUITestOptions() .setPlatformVersion("18") .setDeviceName("iPhone 15") .setApp("tb://YOUR_APP_ID"); options.setCapability("tb:options", Map.of("realDevice", true, "name", "Checkout flow")); IOSDriver driver = new IOSDriver( new URL("https://KEY:SECRET@hub.testingbot.com/wd/hub"), options ); driver.quit(); ``` ```ruby # Run an Appium test on a real Android device in the TestingBot cloud require 'appium_lib' caps = { 'platformName' => 'Android', 'appium:deviceName' => 'Pixel 9', 'appium:platformVersion' => '15', 'appium:app' => 'tb://YOUR_APP_ID', 'tb:options' => { name: 'Checkout flow', realDevice: true } } driver = Appium::Driver.new( { caps: caps, appium_lib: { server_url: 'https://hub.testingbot.com/wd/hub' } }, true ).start_driver ``` ```csharp // Run an Appium test on a real iPhone in the TestingBot cloud var options = new AppiumOptions(); options.PlatformName = "iOS"; options.AddAdditionalAppiumOption("deviceName", "iPhone 15"); options.AddAdditionalAppiumOption("platformVersion", "18"); options.AddAdditionalAppiumOption("app", "tb://YOUR_APP_ID"); options.AddAdditionalAppiumOption("tb:options", new Dictionary { { "realDevice", true }, { "name", "Checkout flow" } }); var driver = new IOSDriver( new Uri("https://KEY:SECRET@hub.testingbot.com/wd/hub"), options ); driver.Quit(); ``` ```php // Run an Appium test on a real Android device in the TestingBot cloud $capabilities = [ 'platformName' => 'Android', 'appium:deviceName' => 'Galaxy S24', 'appium:platformVersion' => '14', 'appium:app' => 'tb://YOUR_APP_ID', 'tb:options' => ['realDevice' => true, 'name' => 'Checkout flow'], ]; $driver = RemoteWebDriver::create( 'https://KEY:SECRET@hub.testingbot.com/wd/hub', $capabilities ); $driver->quit(); ``` Connected · [View documentation](https://testingbot.com/support/app-automate/appium/nodejs) [View documentation](https://testingbot.com/support/app-automate/appium/python) [View documentation](https://testingbot.com/support/app-automate/appium/java) [View documentation](https://testingbot.com/support/app-automate/appium/ruby) [View documentation](https://testingbot.com/support/app-automate/appium/csharp) [View documentation](https://testingbot.com/support/app-automate/appium/php) Same WebDriver protocol you already use, with a single endpoint switch. See the docs for [Appium on TestingBot](https://testingbot.com/support/app-automate/appium) for full examples. ![Replace your in-house device lab](https://testingbot.com/assets/features/appium/devicelab-75f7f33e3f397b019864f68f343211683c6d06a40e9334eda108d9f0d053297d.jpg) Real device cloud ## Replace your in-house device lab Skip buying, hosting and maintaining iOS and Android devices. TestingBot's mobile cloud is on-call 24/7 with hundreds of real, physical devices ready to run your Appium tests. Every Appium session ships with a video screencast, Appium server logs, device logs, screenshots and network HAR. Pull artifacts from our [REST API](https://testingbot.com/support/api) and post them straight into Jira, Slack or your CI dashboard. - Real iPhones, iPads, Galaxy, Pixel & OnePlus - Native, hybrid and mobile web apps - Pristine devices, factory-reset between sessions - HD video, logs and HAR for every test - Upload IPA, APK and TestFlight builds Languages & frameworks ## Appium integrations for every stack Appium uses the W3C WebDriver protocol, so your existing test code drops in with a single hub URL change. We support the official Appium client in every major language. - ### [Java](https://testingbot.com/support/app-automate/appium/java) - [TestNG](https://testingbot.com/support/app-automate/appium/java/testng) - [JUnit](https://testingbot.com/support/app-automate/appium/java) - ### [Python](https://testingbot.com/support/app-automate/appium/python) - [PyTest](https://testingbot.com/support/app-automate/appium/python) - [Robot Framework](https://testingbot.com/support/app-automate/appium/robotframework) - ### [JavaScript](https://testingbot.com/support/app-automate/appium/nodejs) - [WebDriverIO](https://testingbot.com/support/app-automate/appium/nodejs/webdriverio) - [CodeceptJS](https://testingbot.com/support/app-automate/appium/nodejs/codeceptjs) - ### [Ruby](https://testingbot.com/support/app-automate/appium/ruby) - [RSpec](https://testingbot.com/support/app-automate/appium/ruby) - [Cucumber](https://testingbot.com/support/app-automate/appium/ruby) - ### [C#](https://testingbot.com/support/app-automate/appium/csharp) - [NUnit](https://testingbot.com/support/app-automate/appium/csharp/nunit) - [SpecFlow](https://testingbot.com/support/app-automate/appium/csharp/specflow) - ### [PHP](https://testingbot.com/support/app-automate/appium/php) - [PHPUnit](https://testingbot.com/support/app-automate/appium/php) - [Codeception](https://testingbot.com/support/app-automate/appium/php) New to Appium? Read the [Appium quickstart](https://testingbot.com/support/app-automate/appium) or jump straight into a language-specific guide below. Massive parallelism ## Cut mobile suite time from hours to minutes Fan out your Appium suite across hundreds of real devices in parallel. Same test code, a fraction of the wall-clock time, no flakiness from emulator drift. More on how concurrency is sized and priced: [parallel testing](https://testingbot.com/features/parallel-testing). - Up to 100 concurrent Appium sessions - Sessions start in seconds, no device queue - Auto-retry, and your runner's device parallelism works unchanged [Start free](https://testingbot.com/users/sign_up) ![Cut mobile suite time from hours to minutes](https://testingbot.com/assets/features/selenium/parallel-0229d698e59a8a6b92fe50344a08634624f018331ecf8a849ef83cfb35e635a5.png) Spoof device GPS 20+ regions United States United Kingdom Germany France Spain Belgium Australia China Japan Geolocation testing ## Test your app from 20+ regions Spoof the device GPS to anywhere in the world and verify **localisation** , **geo-targeting** and **store-front rules** in iOS App Store, Google Play and your own backend. Secure devices ## Real devices. Wiped after every session. Each Appium session locks one real device for your test only. The moment your test ends, the device is factory-reset, all installed apps are deleted, caches are cleared, and the device is returned to the pool ready for the next customer. GDPR-compliant, with SAML SSO and audit logs for teams. - One real device per session, never shared - Factory-reset between every session - Encrypted app uploads with per-app retention controls [Visit TestingBot Trust Center](https://trust.testingbot.com/) 1. 1 Your test locks one real device A pristine iPhone or Android phone is reserved for your session only. 2. 2 Appium drives real hardware Real sensors, real GPS, real biometrics. Native, hybrid or mobile web. 3. 3 Device factory-reset on release Apps removed, caches cleared, accounts wiped. Returned clean to the pool. Continuous integration ## Plug Appium into your CI/CD in minutes TestingBot integrates with every major CI/CD tool. Trigger Appium suites on every commit and stream device videos and logs back into your build dashboards. ![Jenkins](https://testingbot.com/assets/integrations/jenkins-e586a1c4af833e4c8d38d7b392a4e84bd193f30cae9cf8c9f198a58ec943a1ff.webp) ### [Jenkins](https://testingbot.com/support/integrations/ci-cd/jenkins) ![Bamboo](https://testingbot.com/assets/integrations/bamboo-c8fe09a3cb0b74d6055e3aa6607d77720bfe29ff487b8e3cf825138adf634346.svg) ### [Bamboo](https://testingbot.com/support/integrations/ci-cd/bamboo) ![TeamCity](https://testingbot.com/assets/integrations/teamcity-7e75cdb835903a4983ad276c41d95e569ceac5be486e96ec1f955029ce5bb319.svg) ### [TeamCity](https://testingbot.com/support/integrations/ci-cd/teamcity) ![CircleCI](https://testingbot.com/assets/integrations/circleci-ede029654ff39b60342e4a5bba8fc9a3a49305cc38bda215b7cf8d07e85e0543.webp) ### [CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci) ![GitLab CI](https://testingbot.com/assets/integrations/gitlab-e4c0b6db07d07bc0a400f5606e258f4dc8a51b778e97bbfa51d1d531f3cb049a.webp) ### [GitLab CI](https://testingbot.com/support/integrations/ci-cd/gitlab) ![GitHub](https://testingbot.com/assets/integrations/github-actions-8436242ad9e15381f84f1a98a10dddce15611087f121d6305782e211942b7842.webp) ### [GitHub](https://testingbot.com/support/integrations/ci-cd/github-actions) Device cloud ## Real iPhones, iPads, Galaxy & Pixel devices Test on the iPhones, iPads, Galaxy, Pixel, OnePlus and Xiaomi phones your customers actually use. Latest flagship devices on day one, plus a broad legacy fleet for backwards-compat coverage. [View all devices](https://testingbot.com/support/app-automate/devices) ![Test online on Pixel 8](https://testingbot.com/assets/devices/160h/46-d0ca4c1d1985aefc27c52ba2a1c88748875c574d5358e53adcbdeaee4ed078bb.webp) Pixel 8 ![Test online on Pixel 11](https://testingbot.com/assets/devices/160h/45-b90994534fadde53be99a94521c6ea23a39fdf32ad277240be4e2613540b5019.webp) Pixel 11 ![Test online on iPhone 17 Pro](https://testingbot.com/assets/devices/160h/44-6b4402c6996d9b1620879bb1123c40a6b36abb064e65dca0a1b3f05286779e9c.webp) iPhone 17 Pro ![Test online on iPhone 17 Pro Max](https://testingbot.com/assets/devices/160h/43-59d7e30684332ed4bf25567cbc09ca942eac46d925ccde8352d3e85812480f7d.webp) iPhone 17 Pro Max ![Test online on iPhone 16 Pro](https://testingbot.com/assets/devices/160h/42-de3704079345026847b739a5b1cf5332595904e54ede66abd988a76fb1e6593c.webp) iPhone 16 Pro ![Test online on iPad (9th generation)](https://testingbot.com/assets/devices/160h/41-70f7d18afde7620063a49a76cbfd088f8b59c742f3a5f4dd4a513dd6ec5c676c.webp) iPad (9th generation) Secure Tunnel ## Test apps that talk to internal backends Use our [secure tunnel](https://testingbot.com/support/tunnel) to let your Appium tests reach private APIs, staging environments and services running on localhost. Encrypted end-to-end, no inbound firewall rules required, and ready out of the box for popular CI providers. ![TestingBot secure tunnel for Appium](https://testingbot.com/assets/features/tunnel-diagram-poster-8c25e19e6fe4f7276a46a96b31a0ad0e07bb4f5bf98dc192d2b70897cfbb1a26.png) [Read more](https://testingbot.com/support/tunnel) Buy vs build ## TestingBot vs an in-house device lab Building and maintaining your own iOS and Android device lab is expensive. Devices break, batteries swell, OS updates ship constantly. Here is how a hosted device cloud compares. | Capability | TestingBot | In-house device lab | | --- | --- | --- | | Device coverage | Real iPhones, iPads, Galaxy, Pixel, OnePlus, Xiaomi, refreshed every release | Limited to what you can buy and rack yourself | | Time to first test | Minutes, change one URL in your Appium driver | Weeks of provisioning, MDM and Appium server setup | | Parallel scaling | Up to 100 concurrent sessions, on demand | Capped by the number of devices you own | | Video & logs | Video, device logs, Appium logs and HAR included | Roll your own recording & log aggregation | | Reliability | 99.9% uptime SLA, 24/7 monitoring | You replace dead batteries and broken screens | | Security | Devices factory-reset between every session | Manual cleanup, leftover state risks | | Support | 24/7 human support and dedicated CSM | Stack Overflow and your own logs | FAQ ## Appium on TestingBot: frequently asked questions What is Appium and why run it on TestingBot? Appium is the open-source automation framework for native, hybrid and mobile web apps. TestingBot hosts a fleet of real iPhones, iPads, Galaxy and Pixel devices that you can drive with your existing Appium scripts. Point your driver at `https://hub.testingbot.com/wd/hub` and your test runs on a real device in seconds. Which Appium versions are supported? TestingBot supports Appium 1, Appium 2 and Appium 3, including Appium 2/3 plugins and the latest XCUITest and UIAutomator2 drivers. You can pin a specific Appium server version per-test through capabilities, so you can keep legacy suites running on Appium 1 while migrating new tests to Appium 3. Can I test native, hybrid and mobile web apps? Yes. TestingBot supports all three Appium contexts. Native iOS apps via XCUITest, native Android apps via UIAutomator2 or Espresso, hybrid apps in WebView, and mobile web in Safari and Chrome on real devices. How do I upload my IPA or APK? Upload your build to [TestingBot Storage](https://testingbot.com/support/app-automate/help/upload) via the REST API or our CLI, then reference it from your Appium capabilities. Builds can be retained per-app for fast re-runs. Can I run Appium tests in parallel? Yes. Plans on TestingBot include up to 100 parallel Appium sessions on real devices. Each session locks a dedicated device, so tests do not interfere with one another. Are the devices real or simulators? TestingBot's mobile cloud uses real, physical iOS and Android devices. We also offer iOS Simulators and Android Emulators for early-stage tests, but real devices are the default and the recommended target for accuracy. Can I test apps that need a private backend? Install our [secure tunnel](https://testingbot.com/support/tunnel) and your Appium tests can reach staging APIs, internal services and localhost. No inbound firewall rules needed. Is the device cloud SOC 2 / GDPR compliant? TestingBot is not SOC 2 or ISO 27001 certified. We are GDPR-compliant with EU-only data residency, registered at CSA STAR Level 1 (a published self-assessment), and our control set is modelled on ISO 27001 Annex A. Our datacenter operators hold ISO 27001. Devices are factory-reset between sessions, app uploads are encrypted and SAML SSO and audit logs are available on enterprise plans. ## More reasons to love TestingBot ### 24/7 support Got questions? Ask in the 24/7 in-app customer [chat support](https://testingbot.com#) or [send us an email](https://testingbot.com/contact/new). We are here to help you! TestingBot Support Hey 👋 How can we help? I need support ### Trusted by over 14,000 teams 6M+ Tests 120k+ Users 400+ Enterprises 106 Countries [Request a demo](https://testingbot.com/demo) ### Developer Documentation Everything you need to integrate TestingBot into your workflow. Browse guides, API references and SDKs for web and mobile testing. - [Appium Quickstart](https://testingbot.com/support/app-automate/appium) - [App uploads (IPA / APK)](https://testingbot.com/support/app-automate/help/upload) - [Available devices](https://testingbot.com/support/app-automate/devices) - [CI/CD integration](https://testingbot.com/support/integrations/ci-cd) [Explore Documentation](https://testingbot.com/support) ### Enterprise-grade Security STAR Level 1 Certified GDPR Compliant SAML SSO Single Sign-On 99.9% Uptime SLA [Visit TestingBot Trust Center](https://trust.testingbot.com/) ## Ready to scale your Appium testing? Read our [Appium Cloud Testing](https://testingbot.com/support/app-automate/appium) documentation and run your first test in under 5 minutes. [Start a free trial](https://testingbot.com/users/sign_up) Resources you may like [![Using Touch Actions with Appium](https://testingbot.com/assets/resources/articles/24-490f70ad8a80e4534523707125b88e44e70fe19b126be4f44e689bbd5f6a0cbf.webp)](https://testingbot.com/resources/articles/touch-actions-appium) ### [Using Touch Actions with Appium](https://testingbot.com/resources/articles/touch-actions-appium) How to simulate taps, swipes and pinches on real devices with Appium Touch Actions, with examples, best practices and common pitfalls. [Read more: Using Touch Actions with Appium](https://testingbot.com/resources/articles/touch-actions-appium) [![Appium 2 and Appium 3 Migration Guide](https://testingbot.com/assets/resources/articles/27-121da26036885301ba438e1cdfc81e7ae8045cbc53c0dafaff5b4436000590b1.webp)](https://testingbot.com/resources/articles/appium-2-migration) ### [Appium 2 and Appium 3 Migration Guide](https://testingbot.com/resources/articles/appium-2-migration) What changed between Appium 1, 2 and 3: drivers, capabilities, the removed touch endpoints, and how to upgrade an existing suite. [Read more: Appium 2 and Appium 3 Migration Guide](https://testingbot.com/resources/articles/appium-2-migration) [![Dark Mode Testing with Appium](https://testingbot.com/assets/resources/articles/22-ed15fa0f3fde7566416fff5078b44689dc274439b2fb829f8c68603e909a65d6.webp)](https://testingbot.com/resources/articles/dark-mode-testing) ### [Dark Mode Testing with Appium](https://testingbot.com/resources/articles/dark-mode-testing) How to switch a native app between light and dark mode from an Appium test on iOS and Android, and the layout bugs dark mode tends to expose. [Read more: Dark Mode Testing with Appium](https://testingbot.com/resources/articles/dark-mode-testing) --- URL: https://testingbot.com/test-in-internet-explorer # Internet Explorer Testing Easily test on Internet Explorer Online! Instant access to any IE Browser versions (7-11). Test your website on real IE browsers. No IE Emulators or Simulators. Signup & Start Testing for Free! [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com) ![Superfast, live cross browser testing](https://testingbot.com/assets/environments/svg/ie-075955285f8005808f3d26947aace877f96e5843a33bd4d6e997709d3a294451.svg) Trusted by some of the world's most innovative companies Interactive Testing ## Interactive Testing On Internet Explorer Browsers ### IE 11, IE 10, IE 9, IE 8, IE 7 Run tests on any version of internet explorer, on real Windows desktops. ### Responsive Testing Each test on TestingBot comes with a video screencast of the device, logs and other generated artifacts. ### No more VMs No more VMs to be maintained or software to be installed. TestingBot has you covered. ### +6100 combinations Instant access to any browser on Mac, Linux, Windows and Mobile. Real Browsers ## IE Emulators vs Real Internet Explorer Browsers Internet Explorer emulators do not always show correct results. Test on real IE browsers, running on a real Windows operating system. No Emulators, Simulators or Virtual Machines, try TestingBot and get instant access to a real IE browser. [Start free](https://testingbot.com/users/sign_up) Global Testing ## Geolocation Testing on IE Run Selenium tests from real exit nodes around the world. Verify **geo-targeting** , **geo-blocking** and **localisation** from Australia, Belgium, the United Kingdom, the United States, Germany, France, Spain, China and more. Visual Testing ## Automated IE Screenshots Take automated screenshots of your website on various IE versions. Rapidly test your webpages for visual differences across multiple IE browsers. Secure Tunnel ## Test internal & staging apps in Internet Explorer Use the secure [TestingBot Tunnel](https://testingbot.com/support/tunnel) to quickly and easily test your staging website on any IE browser version. ![TestingBot secure tunnel for IE](https://testingbot.com/assets/features/tunnel-diagram-poster-8c25e19e6fe4f7276a46a96b31a0ad0e07bb4f5bf98dc192d2b70897cfbb1a26.png) [Read more](https://testingbot.com/support/tunnel) FAQ ## Frequently Asked Questions ### Why do I need to test my website on Internet Explorer? Internet Explorer (IE) has a market share of about 5% in the global browser market. Some organizations or websites are designed to only allow IE, while blocking other browsers for security reasons. IE is the default browser on most of the (older) Windows editions, which means a large portion of Windows users will use this browser. ### What is an Internet Explorer (IE) emulator? An Internet Explorer emulator acts as if it is an Internet Explorer browser. It tries to help developers and testers to test their websites on Internet Explorer. Because this is an emulator, it is not accurate enough to use this for testing. TestingBot recommends using its IE browsers running on real desktops, no emulator. ### Why should I not test with an IE emulator? IE Emulators only mimic an Internet Explorer browser screen and User-Agent. It runs slower than a real IE browser and is not as accurate. You cannot accurately test the CSS support of IE with an emulator. If you are using macOS or Linux, your choices are to install a Windows VM or use a cloud service such as TestingBot. ### How does TestingBot help with internet browser testing? TestingBot provides a grid of real browsers, including IE7 up to IE11, which you can use for automated and manual testing. Run a manual IE test on Windows 8, 8.1 or 7. Or perform automated tests on IE7 up to IE11. ### Can I use macOS or Linux to test on IE? If your computer is a macOS or Linux device, we recommend using TestingBot's cloud service to access a IE browser instance. No need to download, install and maintain a Windows VM. Use TestingBot's interactive, browser-based, remote IE service. ### More reasons to love TestingBot ### 24/7 support Got questions? Ask in the 24/7 in-app customer [chat support](https://testingbot.com#) or [send us an email](https://testingbot.com/contact/new). We are here to help you! TestingBot Support Hey 👋 How can we help? I need support ### Trusted by over 14,000 teams 6M+ Tests 120k+ Users 400+ Enterprises 106 Countries [Request a demo](https://testingbot.com/demo) ### Developer Documentation Everything you need to integrate TestingBot into your workflow. Browse guides, API references and SDKs for web and mobile testing. - [Quickstart tutorials for Selenium](https://testingbot.com/support/web-automate/selenium) - [Appium Examples](https://testingbot.com/support/app-automate/appium) - [Integrations](https://testingbot.com/support/integrations) - [List of browsers & devices](https://testingbot.com/support/web-automate/browsers) [Explore Documentation](https://testingbot.com/support) ### Enterprise-grade Security STAR Level 1 Certified GDPR Compliant SAML SSO Single Sign-On 99.9% Uptime SLA [Visit TestingBot Trust Center](https://trust.testingbot.com/) ### Ready to start IE Testing? Start testing your apps with TestingBot. [Start a free trial](https://testingbot.com/users/sign_up) --- URL: https://testingbot.com/test-edge-browser # Online Cross Browser Testing On Microsoft Edge Browser Easily test on Microsoft Edge Online! Make sure your website works on the latest Edge browser versions that come with Microsoft Windows. Test your website on real Edge browsers. Signup & Start Testing for Free! [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) ![Superfast, live cross browser testing](https://testingbot.com/assets/environments/svg/edge-b639cac0c7a843271a0d3a6ccf894cc296064b63b2cd2920209bc5d030f3c70b.svg) Trusted by some of the world's most innovative companies Interactive Testing ## Interactive Testing On Internet Explorer Browsers ### Legacy versions up to the latest version Run tests on various Microsoft Edge versions, on real Windows and macOS desktops. ### Responsive Testing Each test on TestingBot comes with a video screencast of the device, logs and other generated artifacts. ### No more VMs No more VMs to be maintained or software to be installed. TestingBot has you covered. ### +6100 combinations Instant access to multiple Microsoft Edge browsers, on Windows and macOS. Real Browsers ## Microsoft Edge Emulators vs Real Microsoft Edge Browsers Microsoft Edge emulators do not always show correct results. Test on real Microsoft Edge browsers, running on a real Windows operating system. No Emulators, Simulators or Virtual Machines, try TestingBot and get instant access to a real Microsoft Edge browser. [Start free](https://testingbot.com/users/sign_up) ![Superfast, live cross browser testing](https://testingbot.com/assets/environments/svg/edge-b639cac0c7a843271a0d3a6ccf894cc296064b63b2cd2920209bc5d030f3c70b.svg) Global Testing ## Geolocation Testing on Microsoft Edge Perform \geolocation testing\ in various regions across the world. Test Geo Targeting, Geo Blocking and Geo Localization in 10 countries, including Australia, Belgium, United Kingdom, United States, Germany, France, Spain, China and more. Visual Testing ## Automated Microsoft Edge Screenshots Take automated screenshots of your website on various Microsoft Edge versions. Rapidly test your webpages for visual differences across multiple Microsoft Edge browsers. Secure Tunnel ## Test internal & staging apps in Edge Use the secure [TestingBot Tunnel](https://testingbot.com/support/tunnel) to quickly and easily test your staging website on any Microsoft Edge browser version. ![TestingBot secure tunnel for Edge](https://testingbot.com/assets/features/tunnel-diagram-poster-8c25e19e6fe4f7276a46a96b31a0ad0e07bb4f5bf98dc192d2b70897cfbb1a26.png) [Read more](https://testingbot.com/support/tunnel) FAQ ## Frequently Asked Questions ### Why do I need to test my website on Microsoft Edge? Microsoft Edge is a popular browser that comes by default on new Microsoft Windows installations. It has a market percentage of 6% globally. It's important to make sure your website looks and behaves correctly on this web browser. ### How does TestingBot help with Microsoft Edge testing? TestingBot provides an online cloud of Microsoft Edge browsers, running on Windows and macOS machines. You can run manual tests on Edge browsers, or use WebDriver automation to automatically test your website. ### Can I use Microsoft Edge on macOS? TestingBot provides macOS machines equipped with multiple versions of Microsoft Edge. ### More reasons to love TestingBot ### 24/7 support Got questions? Ask in the 24/7 in-app customer [chat support](https://testingbot.com#) or [send us an email](https://testingbot.com/contact/new). We are here to help you! TestingBot Support Hey 👋 How can we help? I need support ### Trusted by over 14,000 teams 6M+ Tests 120k+ Users 400+ Enterprises 106 Countries [Request a demo](https://testingbot.com/demo) ### Developer Documentation Everything you need to integrate TestingBot into your workflow. Browse guides, API references and SDKs for web and mobile testing. - [Quickstart tutorials for Selenium](https://testingbot.com/support/web-automate/selenium) - [Appium Examples](https://testingbot.com/support/app-automate/appium) - [Integrations](https://testingbot.com/support/integrations) - [List of browsers & devices](https://testingbot.com/support/web-automate/browsers) [Explore Documentation](https://testingbot.com/support) ### Enterprise-grade Security STAR Level 1 Certified GDPR Compliant SAML SSO Single Sign-On 99.9% Uptime SLA [Visit TestingBot Trust Center](https://trust.testingbot.com/) ### Ready to start Microsoft Edge Testing? Start testing your apps with TestingBot. [Start a free trial](https://testingbot.com/users/sign_up) --- URL: https://testingbot.com/test-on-safari Safari cloud # Test on real Safari, online in the cloud Run your manual, automated and visual tests on real Safari, from Safari 9 through Safari 27 on macOS, plus Mobile Safari on real iPhones and iPads from iOS 9 to 27. Nobody emulates Safari convincingly, so desktop Safari here is always the real browser on real Apple hardware; for Mobile Safari you can also start on a hosted iOS simulator when you want a fast check. [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) Free plan, no credit card: 2 live sessions on Safari 27 (macOS Golden Gate). Every Safari and macOS version below is on the paid plans and the 7-day full-access trial. - **Safari versions:** 9 → 27 - **macOS + iOS devices:** Real - **Uptime SLA:** 99.9% ![Safari](https://testingbot.com/assets/environments/svg/safari-e4837bb5f9a9729fef843af4180372e975c62587808b8ab479f615f3e240e858.svg) Trusted by some of the world's most innovative companies 30-second answer ## Why test Safari on TestingBot? Real Safari, real Apple hardware TestingBot streams real Safari running on real macOS and real iPhones / iPads, so there is no emulated WebKit and no "looks-like-Safari" surprises. Hosted iOS simulators are available too when you want a quicker Mobile Safari check. Every Safari since Safari 9 Test on every Safari version that still ships, from Safari 9 on El Capitan through Safari 27 on Golden Gate. Match what your customers actually use. Manual, automated, visual Drive Safari live from your browser, automate it via Selenium, Playwright or Appium, or capture screenshots for visual regression, same grid, same login. What is Safari? ## Safari in 60 seconds ![Safari](https://testingbot.com/assets/environments/svg/safari-e4837bb5f9a9729fef843af4180372e975c62587808b8ab479f615f3e240e858.svg) ### Safari Apple · WebKit · macOS, iOS, iPadOS Safari is Apple's web browser, built on the WebKit rendering engine. It is the default browser on every macOS, iOS and iPadOS device, and it is the only engine permitted on iOS, so any iPhone or iPad user, on any browser, is effectively running Safari. That makes Safari coverage non-optional for any consumer-facing site. WebKit's quirks around CSS, ITP (Intelligent Tracking Prevention), date/time inputs and PWAs catch teams off guard if they only test in Chromium during development. - Built on WebKit, maintained by Apple - Default browser on macOS, iOS and iPadOS - ITP, ATS and the strictest cookie policy on the web - Only browser engine allowed on iOS - Safari Technology Preview & Safari Beta available What you can do ## Everything Safari testing needs Live debugging, automated CI runs, visual regression, extension testing, geolocation — all on the same fleet of real Apple devices. - ### Live & manual testing Drive Safari interactively from your browser. Native Web Inspector, network panel, console — same as if you were on a real Mac. - ### Automated testing Run Selenium, Playwright or Appium suites against Safari in parallel. Native Safaridriver on desktop, Appium on real iPhones and iPads. - ### Visual & screenshot testing Take pixel-perfect screenshots across every Safari version, then diff them automatically with our visual testing service. - ### Extension testing Load unsigned Safari App Extensions, Web Extensions (Xcode) and Safari Web Extensions 18.4+ in manual sessions and CI. - ### Geolocation & ITP Test from 20+ regions and validate Safari's Intelligent Tracking Prevention behaviour on real exit nodes. - ### Tech Preview & Beta Run today's site against tomorrow's Safari. Both Safari Technology Preview and Safari Beta are available alongside stable releases. Safari versions ## Test On Different Safari Browsers Online Real Safari shipping with real macOS releases — and Mobile Safari on real iPhones and iPads. ### macOS Desktop - macOS Golden Gate Safari 27 - macOS Tahoe Safari 26 - macOS Sequoia Safari 18 - macOS Sonoma Safari 18 - macOS Ventura Safari 16 - macOS Monterey Safari 15 - macOS Big Sur Safari 14 - macOS Catalina Safari 13 - macOS Mojave Safari 12 - macOS High Sierra Safari 11 - OS X El Capitan Safari 9 ![iOS Safari](https://testingbot.com/assets/environments/svg/ios-383468addf160fa18d0e431f529420739d7f7d1206175f682fead627d2e99a52.svg) ### Mobile Safari - iOS 27 - iOS 26 - iOS 18 - iOS 17 - iOS 16 - iOS 15 - iOS 14 - iOS 13 - iOS 12 - iOS 11 - iOS 10 - iOS 9 iOS simulator vs real Apple hardware ## When an iOS simulator is enough, and when you need real Apple hardware If you searched for a Safari emulator, what exists in practice is Apple's iOS Simulator, and TestingBot hosts those for quick Mobile Safari checks. Nothing emulates desktop Safari convincingly, and a simulator's WebKit drifts from the one Apple ships, so release checks belong on real Apple hardware. Here is where the two differ. | Capability | TestingBot · real Safari | WebKit emulator / simulator | | --- | --- | --- | | Browser engine | Real Safari, real WebKit shipped by Apple | Open-source WebKit, drifts from shipped Safari | | Operating system | Real macOS El Capitan through macOS Golden Gate | Linux container or your dev machine | | Mobile Safari | Real iPhones and iPads from iOS 9 to 27 | Mobile-emulated user agent only | | ITP & cookie policy | Real Apple ITP behaviour | Inconsistent or absent | | Safari extensions | Load unsigned Safari Web Extensions | Not supported | | Version coverage | Safari 9 → 27, plus Tech Preview & Beta | Latest WebKit only | Live exit nodes 10+ countries United States United Kingdom Germany France Spain Belgium Australia China Japan Geolocation testing ## Geolocation Testing on Safari Test your website with Safari from different regions in the world. Secure Tunnel ## Test internal & staging apps in Safari Use the secure [TestingBot Tunnel](https://testingbot.com/support/tunnel) to quickly and easily test your staging website on any Safari browser version on macOS and iOS. ![TestingBot secure tunnel for Safari](https://testingbot.com/assets/features/tunnel-diagram-poster-8c25e19e6fe4f7276a46a96b31a0ad0e07bb4f5bf98dc192d2b70897cfbb1a26.png) [Read more](https://testingbot.com/support/tunnel) Mobile Safari · Appium ## Automated Mobile Safari Testing with Appium Use [Appium](https://testingbot.com/features/automation/appium) to connect to our grid of physical iOS devices and run automated mobile Safari tests against your website. [Read more about Appium mobile testing](https://testingbot.com/features/automation/appium) ![Appium](https://testingbot.com/assets/integrations/appium-75e88eb18ca3b2ce63d641547ce06398c8ecd971ed17187c9134c8d205465f18.svg) ![Mobile Safari](https://testingbot.com/assets/environments/svg/safari-e4837bb5f9a9729fef843af4180372e975c62587808b8ab479f615f3e240e858.svg) Same Appium APIs, real Mobile Safari on real iPhones. Test from Windows ## Test Safari on Windows, no Mac required On Windows and need to test something in Safari? You don't have to buy a Mac, set up a virtual machine or chase down a borrowed laptop. TestingBot streams a real macOS desktop straight to your browser, so you can drive any Safari version: from Safari 9 on El Capitan through Safari 27 on Golden Gate, directly from Windows 11, 10, or any older PC. Looking for a Safari emulator for Windows? TestingBot is the better alternative, with real Safari running on real macOS instead of a Windows-based simulator that drifts from what Apple ships. Open the Web Inspector, switch tabs, validate Intelligent Tracking Prevention, run extensions, every developer tool the real Safari ships with, available from Edge, Chrome or Firefox on your Windows machine. [Start free](https://testingbot.com/users/sign_up) Windows ![Safari on macOS](https://testingbot.com/assets/environments/svg/safari-e4837bb5f9a9729fef843af4180372e975c62587808b8ab479f615f3e240e858.svg) Real macOS No VM to set up No Mac to buy Real Safari dev tools FAQ ## Frequently Asked Questions Which Safari versions are supported? TestingBot supports every Safari from Safari 9 (OS X El Capitan) through Safari 27 (macOS Golden Gate), plus Safari Technology Preview and Safari Beta. On mobile, Mobile Safari is available on real iPhones and iPads from iOS 9 through iOS 27. Can I use Safari on Windows? Apple discontinued Safari for Windows in 2012, so there is no modern Safari version that installs natively on a Windows PC. The supported way to test on Safari from Windows today is to drive a real macOS desktop in the cloud. With TestingBot you can stream a real Mac running any Safari version from Safari 9 through Safari 27 directly to Windows 11, 10 or older, with no virtual machine and no Mac purchase. If you were searching for a Safari emulator for Windows, the closest honest equivalent is real Safari on real macOS, streamed to your PC, rather than a WebKit simulation that drifts from what Apple ships. How can I test Safari without a Mac? TestingBot streams real macOS desktops to any Windows or Linux browser, so you can test on Safari without owning or maintaining a Mac. You get the actual Safari developer tools, real WebKit rendering and full version coverage from Safari 9 to Safari 27, a cloud replacement for both a Safari emulator and an in-house Mac fleet. Why do I need to test my website on Safari? Safari has a market share of about 4% in the global browser market. This browser is built and maintained by Apple and supports all the latest browser technologies. Safari is the default browser on all macOS operating systems, which means a large portion of macOS users will use this browser for their day-to-day browsing. Test Safari's private browsing capabilities, open multiple tabs or test the cross site tracking prevention. How can I automate Mobile Safari? We recommend using Appium, which has built-in support to communicate with iOS Safari. Simply run your Appium scripts against the TestingBot device grid. TestingBot will instruct the iOS device to open Safari. Your Appium script will then instrument the mobile Safari browser in an automated way. What is Safaridriver? Safaridriver is a utility, shipped by default in every macOS operating system, which allows Selenium to automate Safari through a HTTP based API. To use Safaridriver, you need to make sure it is enabled by running this command in a terminal window: `safaridriver --enable` How can I test on Mobile Safari? TestingBot provides access to both iOS simulators and physical iOS devices. Use the developer tools to inspect webpages running on Mobile Safari, straight from TestingBot's Manual Testing service. Can I test Safari on iPhone and iPad? TestingBot provides access to physical iOS devices, located in our datacenter. You can run your automated or manual tests against iOS Mobile Safari on our iPhones and iPads. How to test on older Safari versions? TestingBot provides access to Safari versions from Safari 9 up to Safari 27. Why is TestingBot the preferred choice for Safari Testing? TestingBot provides real Safari browsers, ranging from Safari 9 up to Safari 27. You can use the developer tools that come with Safari to debug webpages, inspect elements and run performance tests. You can test the intelligent tracking prevention, test Safari extensions, open multiple tabs and perform all other actions on a remote Safari instance. ## Ready to start Safari Testing? Get started with Safari testing today. Run your tests on real Safari browsers and real Apple devices in the TestingBot cloud. [Start a free trial](https://testingbot.com/users/sign_up) Resources you may like [![Test on Safari Tech Preview and Safari beta](https://testingbot.com/assets/blog/2023/safari-testing.png)](https://testingbot.com/blog/safari-tech-preview-beta) ### [Test on Safari Tech Preview and Safari beta](https://testingbot.com/blog/safari-tech-preview-beta) Run Automated and Manual tests on Safari Technology Preview and Safari Beta with TestingBot, on macOS Monterey and macOS Ventura. [Read more: Test on Safari Tech Preview and Safari beta](https://testingbot.com/blog/safari-tech-preview-beta) [![Test your website and mobile apps on iOS 18.](https://testingbot.com/assets/blog/2024/ios18.webp)](https://testingbot.com/blog/ios-18-testing) ### [Test your website and mobile apps on iOS 18.](https://testingbot.com/blog/ios-18-testing) Ready to start testing on iOS 18? Get started today with TestingBot: run automated, visual and manual tests on iOS 18. [Read more: Test your website and mobile apps on iOS 18.](https://testingbot.com/blog/ios-18-testing) [![Safari Technology Preview: Automated Testing with WebDriver](https://testingbot.com/assets/blog/2016/06/safari_for_developers_-_logo_thumb800-300x150.jpg)](https://testingbot.com/blog/2016/06/14/safari-technology-preview-automated-testing-with-webdriver) ### [Safari Technology Preview: Automated Testing with WebDriver](https://testingbot.com/blog/2016/06/14/safari-technology-preview-automated-testing-with-webdriver) [Read more: Safari Technology Preview: Automated Testing with WebDriver](https://testingbot.com/blog/2016/06/14/safari-technology-preview-automated-testing-with-webdriver) --- URL: https://testingbot.com/test-on-chrome Chrome cloud # Test on real Chrome, online in the cloud An online Chrome browser for manual, automated, responsive and visual testing. Desktop Chrome runs as real, unmodified Google Chrome on real operating systems; for mobile Chrome you can pick a real Android phone or a hosted Android emulator. From Chrome 35 to the latest, plus Chrome Beta, Dev and Canary. [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) - **Chrome versions:** 35 → latest - **Platforms:** Win · macOS · Linux · Android - **Uptime SLA:** 99.9% ![Google Chrome](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Trusted by some of the world's most innovative companies 30-second answer ## Why test Chrome on TestingBot? Real desktop Chrome, plus Android emulators On desktop, TestingBot streams unmodified Google Chrome straight from Google on real Windows, macOS and Linux. No headless mode and no Chromium drift. For mobile Chrome, choose a real Android phone when you need hardware fidelity, or a hosted Android emulator when you want a fast check. Stable, Beta, Dev, Canary Test on Chrome Stable, Chrome Beta, Chrome Dev and Chrome Canary side by side. Catch regressions before Google rolls them out to your customers. Manual, automated, responsive, visual Drive Chrome live from your browser, resize on the fly for responsive testing, automate via Selenium, Playwright or Puppeteer, or capture screenshots for visual regression. Same grid, same login. What you can do ## Everything Chrome testing needs Live debugging, automated CI runs, responsive testing, visual regression, extension testing, geolocation, ChromeOS, performance profiling. All on the same fleet of real Chrome browsers. - ### Live & manual testing Drive Chrome interactively from your browser. Native DevTools, console, network panel, same as if you were on your own machine. - ### Automated testing Run Selenium, Playwright, Puppeteer or Cypress against Chrome in parallel. Native Chromedriver auto-matched to every browser version. - ### Responsive testing Resize the browser on the fly during a manual session, or set any viewport from your automation script. Validate breakpoints, mobile menus and fluid layouts on real Chrome. - ### Visual & screenshot testing Take pixel-perfect screenshots across every Chrome version, then diff them automatically with the visual testing service. - ### Extension testing Side-load unpacked Chrome extensions and CRX bundles in manual sessions and automated runs. Manifest v2 and v3 supported. - ### Performance & Lighthouse Capture HAR files, Lighthouse reports and DevTools performance traces. Throttle CPU and network to validate slow-3G behaviour. - ### Camera, microphone & WebGL Test getUserMedia with dummy camera and microphone streams, plus full WebGL support for 3D and canvas-heavy apps. - ### ChromeOS testing Run a ChromeOS emulator hosted on an Android emulator to verify Chromebook behaviour, ChromeOS-specific Chrome quirks and progressive web apps. A practical Chrome OS emulator when you do not have a Chromebook on your desk. Chrome coverage ## Test On Different Chrome Versions Online Real Chrome on real Windows, macOS and Linux desktops, plus Chrome on real Android phones. ### Channels & versions - Chrome Stable latest - Chrome Beta next - Chrome Dev future - Chrome Canary daily - Older majors 35 → latest - Auto-updated weekly Google ### Platforms - Windows 11 - Windows 10 - macOS Tahoe - macOS Sequoia - macOS Sonoma - macOS Ventura - Linux - Real Android phones - ChromeOS (Android emulator) Chrome emulator vs real Chrome ## When a Chrome emulator is enough, and when you need real Chrome An emulated or headless Chrome is fine for a fast pass on layout and obvious breakage, and TestingBot hosts Android and ChromeOS emulators for exactly that. Real Chrome on a real operating system is what you want when GPU behaviour, extensions, media playback or the exact shipped Chrome binary matter. Here is where the two differ. | Capability | TestingBot · real Chrome | Chrome emulator / headless | | --- | --- | --- | | Browser binary | Real Google Chrome, signed by Google | Headless Chrome, Chromium build or browser-based emulator | | Operating system | Real Windows, macOS and Linux | One Linux container or your dev machine | | Mobile Chrome | Real Chrome on real Android phones | Mobile-emulated user agent only | | Version coverage | Chrome 35 → latest, plus Beta, Dev and Canary | Whatever Chrome happens to be installed | | Camera, mic & WebGL | Dummy media streams and full WebGL support | Often disabled or stubbed | | Video, logs & HAR | HD video, browser logs, HAR and screenshots | Roll your own recording and capture | Live exit nodes 10+ countries United States United Kingdom Germany France Spain Belgium Australia China Japan Geolocation testing ## Geolocation Testing on Chrome Run Chrome from real exit nodes around the world to verify geo-targeting, geo-blocking and localisation from the United States, United Kingdom, Germany, France, Spain, Belgium, Australia, China, Japan and more. Secure Tunnel ## Test internal & staging Chrome apps securely Use our [secure tunnel](https://testingbot.com/support/tunnel) to run Chrome tests against firewalled staging sites, internal dashboards or apps still on localhost. Encrypted end-to-end, no inbound firewall rules required, and built-in support for popular CI providers. ![TestingBot secure tunnel for Chrome](https://testingbot.com/assets/features/tunnel-diagram-poster-8c25e19e6fe4f7276a46a96b31a0ad0e07bb4f5bf98dc192d2b70897cfbb1a26.png) [Read more](https://testingbot.com/support/tunnel) Mobile Chrome ## Real Chrome on real Android devices Drive Chrome on real Pixel, Galaxy, OnePlus and Xiaomi devices using Selenium WebDriver, Playwright or Appium. Real hardware, real sensors, accurate results. [View all devices](https://testingbot.com/support/app-automate/devices) ![Chrome](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) ![Android](https://testingbot.com/assets/environments/svg/android-1b3d235e17522e2238e1c55b4e9cdd41219f653cd958ea065877d3813f32ffc3.svg) Same WebDriver APIs, real Chrome on real Android phones. Camera, Microphone & WebGL ## Camera, Microphone and WebGL Testing on Chrome Test getUserMedia flows with a dummy camera and microphone in real Chrome. No plug-ins, no extra permissions to wire up. Run WebGL tests in the cloud against the same GPU stack Chrome ships to your customers. ### Dummy camera Drive Chrome with a fake-camera flag so getUserMedia returns a deterministic video stream. Perfect for testing video calls, KYC and webcam-test pages without real hardware. ### Dummy microphone Use a synthetic audio source for getUserMedia, ideal for chrome microphone test flows, voice recording, audio chat and accessibility scenarios. No real mic required. ### Full WebGL support Run real WebGL tests in Chrome with hardware-accelerated graphics. 3D, canvas-heavy SPAs and games render exactly as they do on a real Chromebook or PC. Test from anywhere ## Test Chrome from any OS, even old PCs Don't have a clean Chrome install? Stuck on a corporate VM, an old PC or a Mac that won't update? TestingBot streams a real desktop running Chrome on Windows 11, 10, macOS or Linux directly to any browser, so you can test on any Chrome version from any device. No installs, no admin rights required. The streamed Chrome session is the real thing, with full DevTools, profile switching, extension support and Lighthouse audits. You drive it from any browser on your local machine, without installing Chrome or running it locally. [Start free](https://testingbot.com/users/sign_up) No install required No admin rights Real Chrome DevTools FAQ ## Frequently Asked Questions Which Chrome versions are supported? TestingBot supports every Chrome from Chrome 35 through the current Stable, refreshed within hours of each Google release. Chrome Beta, Chrome Dev and Chrome Canary are available alongside Stable, on Windows, macOS, Linux and real Android devices. Why do I need to test my website on Chrome? Chrome is the most popular web browser on the planet, with over 60% market share globally. You want to make sure that your website looks and functions correctly on this browser. The browser is built and maintained by Google and is used all around the world, on Windows, macOS, Linux, ChromeOS and Android. Test cookie behaviour, ad-blockers, the third-party cookie phase-out and Chrome's progressive Privacy Sandbox features. What is the difference between Chrome, Chromium, Chrome Beta and Chrome Canary? Chrome (Stable) is the browser everyone uses for day-to-day browsing. It is the same binary TestingBot provides for testing. Chromium uses the same browser engine but ships without Google's branding and proprietary components, and it is open-source. Chrome Beta is the next major Chrome release, roughly 4 to 6 weeks ahead of Stable. Chrome Dev sits one step further out, and Chrome Canary is the bleeding-edge daily build. TestingBot lets you run all four channels side by side so you can spot regressions before Google promotes them to Stable. What is Chromedriver? Chromedriver is a program that automates the Chrome browser through the W3C WebDriver protocol. Selenium, Puppeteer (over the Chrome DevTools Protocol) and Playwright all use it to drive Chrome. Chromedriver versions are tightly coupled to Chrome versions, which means you need to make sure the Chromedriver version matches the browser version. At TestingBot this is handled automatically. Pick a Chrome version and the matching Chromedriver is provisioned for you. Can I test Chrome on real Android devices? Yes. TestingBot ships real Pixel, Galaxy, OnePlus and Xiaomi devices running real Chrome for Android. Drive them with Selenium WebDriver or Appium and use the actual mobile rendering engine, mobile DevTools and real touch input. Does TestingBot offer a Chrome emulator, or real Chrome? Both, and the distinction matters. Desktop Chrome on TestingBot is real Chrome, downloaded straight from Google and running on real Windows, macOS or Linux, not a headless container. For mobile Chrome we offer real Android phones and hosted Android emulators, plus a ChromeOS emulator for Chromebook behaviour. Sessions run inside dedicated single-use virtual machines on Windows, macOS or Linux, with full GPU acceleration, real Chrome DevTools and the same Chrome binary your customers use. If you have been searching for an "online Chrome emulator" or a "Chrome browser online" tool to validate a site, TestingBot gives you the real thing instead. Can I test Chrome extensions? Yes. Side-load unpacked extensions and CRX bundles in manual sessions for click-through testing, or load them automatically in Selenium, Playwright and Puppeteer runs. Manifest v2 and v3 are both supported. Can I run Lighthouse on TestingBot? Yes. Run Lighthouse audits inside any Chrome session, capture the JSON report as a build artifact, and trend Core Web Vitals over time. Throttle CPU and network from the test options to simulate slow connections. ## Ready to start Chrome Testing? Start testing your website on real Chrome in the cloud. Every version, every platform, every release channel. [Start a free trial](https://testingbot.com/users/sign_up) --- URL: https://testingbot.com/test-on-iphone Real iPhone cloud # iPhone testing online, on simulators or real iPhones Run manual, automated, visual and responsive tests on iPhone in the TestingBot cloud. Start on an iOS simulator when you want a fast first signal, then run the same tests on real iPhones before you ship. From iPhone 6s through iPhone 17, with real Mobile Safari, real iOS apps and real touch input. Nothing to install, no Mac and no jailbroken devices. [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) - **iPhone models:** 6s to 17 - **iOS versions:** 12 to 27 - **Uptime SLA:** 99.9% _A real iPhone running Mobile Safari with animated tap gestures_ Trusted by some of the world's most innovative companies 30-second answer ## Why test on real iPhones with TestingBot? iOS simulators and real iPhones Launch a hosted iOS simulator in seconds for quick checks, or connect to a physical, unaltered iPhone in our datacenter when you need real Apple silicon, real Mobile Safari and real touch and motion sensors. Both run from the same browser session, with no Mac and no Xcode to install. Every iPhone since the 6s Pick from iPhone 6s, SE, 7, 8, X, XR, XS, 11, 12, 13, 14, 15, 16 and 17 across iOS 12 through iOS 27. Match what your customers actually carry in their pockets. Manual, automated, app & web Drive an iPhone live from your browser, run Appium suites against the device cloud, capture screenshots for visual regression, or upload your IPA and exercise the real iOS app. What you can do ## Everything iPhone testing needs Live debugging, automated Appium runs, mobile-web and native-app testing, gestures, network throttling, geolocation. All on a fleet of real iPhones in our datacenter. - ### Live & manual testing Take an iPhone in your browser. Tap, swipe, type, rotate and inspect, with full DevTools through Mobile Safari. Same as if you were holding the phone in your hand. - ### Appium automation Run Appium 1, 2 and 3 suites against real iPhones in parallel. WebDriverAgent, XCUITest and full XCTest support, no Mac required on your side. - ### Native iOS app testing Upload your IPA, install it in seconds, and exercise it on real iPhones. App permissions, biometrics, keyboards, push and deep links all behave like real hardware. - ### Real Mobile Safari Test your website on Mobile Safari, the only browser engine Apple permits on iOS, on real Apple hardware. The Safari your iPhone customers actually use. - ### Visual & screenshot testing Capture pixel-perfect screenshots across every iPhone model and iOS version, then diff them automatically with the visual testing service. - ### Tap, swipe, pinch and rotate Drive every multi-touch gesture exactly as a customer would, including pinch zoom, long-press, scroll, three-finger swipes and device rotation. - ### Network throttling Test on WiFi, 5G, 4G, 3G, Edge and offline conditions. Catch slow-network bugs before App Store reviewers do. - ### Responsive testing Switch between portrait and landscape, swap between iPhone Pro Max and iPhone SE in one click, and validate breakpoints on real Safari. Device coverage ## Test On Different iPhones Online Real iPhones from iPhone 6s through the latest iPhone 17, plus iPad. Real Apple silicon, real Mobile Safari, real touch input. ### iPhone models - [iPhone 17 Pro Max](https://testingbot.com/test-on-iphone/apple-iphone-17-pro-max) - [iPhone 17 Pro](https://testingbot.com/test-on-iphone/apple-iphone-17-pro) - [iPhone 16 Pro Max](https://testingbot.com/test-on-iphone/apple-iphone-16-pro-max) - [iPhone 16](https://testingbot.com/test-on-iphone/apple-iphone-16) - [iPhone 15 Pro](https://testingbot.com/test-on-iphone/apple-iphone-15-pro) - [iPhone 15](https://testingbot.com/test-on-iphone/apple-iphone-15) - [iPhone 14 Pro](https://testingbot.com/test-on-iphone/apple-iphone-14-pro) - [iPhone 14](https://testingbot.com/test-on-iphone/apple-iphone-14) - [iPhone 13 Pro Max](https://testingbot.com/test-on-iphone/apple-iphone-13-pro-max) - [iPhone 13 Pro](https://testingbot.com/test-on-iphone/apple-iphone-13-pro) - [iPhone 13](https://testingbot.com/test-on-iphone/apple-iphone-13) - [iPhone 13 mini](https://testingbot.com/test-on-iphone/apple-iphone-13-mini) - [iPhone 12 Pro Max](https://testingbot.com/test-on-iphone/apple-iphone-12-pro-max) - [iPhone 12](https://testingbot.com/test-on-iphone/apple-iphone-12) - [iPhone 12 mini](https://testingbot.com/test-on-iphone/apple-iphone-12-mini) - [iPhone 11 Pro Max](https://testingbot.com/test-on-iphone/apple-iphone-11-pro-max) - [iPhone 11 Pro](https://testingbot.com/test-on-iphone/apple-iphone-11-pro) - [iPhone 11](https://testingbot.com/test-on-iphone/apple-iphone-11) - [iPhone XR](https://testingbot.com/test-on-iphone/apple-iphone-xr) - [iPhone XS Max](https://testingbot.com/test-on-iphone/apple-iphone-xs-max) - [iPhone X](https://testingbot.com/test-on-iphone/apple-iphone-x) - [iPhone SE (2022)](https://testingbot.com/test-on-iphone/apple-iphone-se-2022) - [iPhone SE (2020)](https://testingbot.com/test-on-iphone/apple-iphone-se-2020) - [iPhone 8 Plus](https://testingbot.com/test-on-iphone/apple-iphone-8-plus) - [iPhone 8](https://testingbot.com/test-on-iphone/apple-iphone-8) - [iPhone 7](https://testingbot.com/test-on-iphone/apple-iphone-7) - [iPhone 6s Plus](https://testingbot.com/test-on-iphone/apple-iphone-6s-plus) - [iPhone 6s](https://testingbot.com/test-on-iphone/apple-iphone-6s) [View all iPhones and iPads](https://testingbot.com/support/app-automate/devices) ![iOS](https://testingbot.com/assets/environments/svg/ios-383468addf160fa18d0e431f529420739d7f7d1206175f682fead627d2e99a52.svg) ### iOS versions - iOS 27 - iOS 26 - iOS 18 - iOS 17 - iOS 16 - iOS 15 - iOS 14 - iOS 13 - iOS 12 Latest iOS releases land on the cloud within hours of Apple shipping them. Older majors stay around for legacy regression coverage. New · iOS 27.1 simulator ### iPhone Duo, before it ships Apple has not released the iPhone Duo yet, but you can already test on it as an iOS 27.1 simulator. See both the cover screen and the inner screen, fold and unfold with one click, and debug with Safari Web Inspector. It runs as a simulator, not a real device. [Start a live test on iPhone Duo](https://testingbot.com/members/manual/new) iOS simulator vs real iPhone ## When an iOS simulator is enough, and when you need a real iPhone TestingBot gives you both. An iOS simulator is the right tool for a fast pass on layout, navigation and obvious breakage, and it is cheap to run on every commit. A real iPhone is what you want before release, because a simulator cannot reproduce Apple silicon performance, the shipped WebKit, biometrics or real touch. Here is where the two differ. | Capability | TestingBot, real iPhones | iOS simulator | | --- | --- | --- | | Hardware | Real Apple silicon, real iPhone in our datacenter | x86 or ARM Mac, no real Apple radios or sensors | | Mobile Safari | Real WebKit shipped on the device by Apple | Simulator WebKit, drifts from shipped Safari | | Touch & gestures | Real multi-touch, real haptics, real rotation | Mouse-faked taps, no haptics | | Native apps | Real iOS, real frameworks, real ClassKit and StoreKit | Some Apple frameworks missing or stubbed | | Device coverage | iPhone 6s through iPhone 17, iOS 12 to 27 | Hosted iOS simulator images, no Mac or Xcode install | | Video, logs and HAR | HD video, device logs, HAR and screenshots | Roll your own recording and capture | Native iOS apps ## Native iOS app testing Upload your IPA and TestingBot installs it on a real iPhone in seconds. Drive the app live or via Appium, capture full HD video, syslog and crash reports, and exercise biometrics, push notifications, deep links and the iOS keyboard exactly as your customers will. Every iPhone above is physical hardware racked in our European datacenter, not a simulator. The [real device testing](https://testingbot.com/real-device-testing) overview lists the full fleet, explains how each handset is wiped and re-imaged between sessions, and shows which plans include it. [Read more](https://testingbot.com/features/automation/appium) upload-and-test.js // 1) Upload your IPA to TestingBot storageconst TestingBot = require('testingbot-api'); const tb = new TestingBot({ api\_key: 'YOUR\_KEY', api\_secret: 'YOUR\_SECRET' }); tb.uploadFile('./MyApp.ipa', (err, res) =\> { // res.app\_url → "tb://..."// 2) Run your Appium test on a real iPhone driver.init({ platformName: 'iOS', 'appium:deviceName': 'iPhone 17', 'appium:platformVersion': '27', 'appium:realDevice': true, 'appium:app': res.app\_url }); }); Live exit nodes 10+ countries United States United Kingdom Germany France Spain Belgium Australia China Japan Geolocation testing ## Geolocation Testing on iPhones Run iPhone tests from real exit nodes around the world to verify geo-targeting, geo-blocking and localisation from the United States, United Kingdom, Germany, France, Spain, Belgium, Australia, China, Japan and more. Secure Tunnel ## Test internal & staging sites on real iPhones Use our [secure tunnel](https://testingbot.com/support/tunnel) to run iPhone tests against firewalled staging sites, internal dashboards or apps still on localhost. Encrypted end-to-end, no inbound firewall rules required, ready out of the box for popular CI providers. ![TestingBot secure tunnel for iPhone](https://testingbot.com/assets/features/tunnel-diagram-poster-8c25e19e6fe4f7276a46a96b31a0ad0e07bb4f5bf98dc192d2b70897cfbb1a26.png) [Read more](https://testingbot.com/support/tunnel) Secure devices ## Real iPhones. Wiped after every session. Each session locks a single iPhone for your test only. The moment your test ends, the device is wiped, all installed apps are deleted, caches are cleared, and the iPhone is returned to the pool ready for the next customer. GDPR-compliant, with SAML SSO and audit logs available for enterprise teams. - One real iPhone per session, never shared - Wiped between every session - Encrypted IPA uploads with per-app retention controls 1 customer per session Datacenter hosted Wiped after each test CSA STAR Level 1 GDPR compliant SAML SSO enterprise Test without a Mac ## Test on iPhones from any OS, no Mac required On Windows or Linux and need to validate something on an iPhone? You don't have to buy a Mac, set up Xcode, or chase down a borrowed iPad. TestingBot streams a real iPhone from our datacenter directly to any browser, so you can drive a physical iPhone from Windows 11, 10, Linux or any older PC. The streamed session is the real device, with Mobile Safari, the iOS keyboard, biometrics and notifications. You drive it from any browser on your local machine, without installing Xcode or owning Apple hardware. [Start free](https://testingbot.com/users/sign_up) No Mac to buy No Xcode to install Real iPhone in your browser FAQ ## Frequently Asked Questions Which iPhones and iOS versions are supported? TestingBot supports physical iPhones from iPhone 6s through iPhone 17, including the Pro and Pro Max line, plus the iPhone SE family. iOS versions range from iOS 12 through iOS 27. New flagship phones are added on or close to the day Apple ships them. How do I test a mobile app on iPhone? Testing your iOS app on a variety of iOS devices is important before publishing to the App Store. TestingBot lets you upload your IPA and install it on a number of different iPhones in our datacenter. Choose between various iOS versions and screen sizes to make sure your mobile app works correctly on every iPhone your customers might own. Why is testing your website on an iPhone necessary? iPhone has roughly a 28% global mobile market share, much higher in the United States, the United Kingdom, Japan and Australia. Your website needs to look and behave correctly on Mobile Safari for those users. Mobile Safari is the only browser engine Apple permits on iOS, so testing on Chrome, Firefox or any other browser does not represent the experience your iPhone customers actually have. Should I use an iOS simulator or a real iPhone? Use both, at different points. An iOS simulator is fast to launch and cheap to run on every commit, and it is perfectly good for catching layout breakage, broken navigation and obvious regressions. TestingBot hosts iOS simulators so you can do that without a Mac or an Xcode install. Before you ship, move to a real iPhone. A simulator does not match real device performance, several Apple frameworks (StoreKit, ClassKit, biometrics, push) are missing or stubbed, and the WebKit it ships drifts from the WebKit on a retail iPhone. The practical split is simulators for speed during development, real devices for release confidence. Can I test Mobile Safari on a real iPhone? Yes. TestingBot provides physical, unaltered iPhones, straight from the store and never jailbroken. Run your manual or automated tests against real Mobile Safari on a physical iPhone in our datacenter, with full Web Inspector, network panel and console. Can I test on iPhone without owning a Mac? Yes. TestingBot streams real iPhones to any Windows, macOS or Linux browser. You get the real device, real Mobile Safari and real iOS, without owning a Mac, an iPhone or a copy of Xcode. Can I run Appium against your iPhone cloud? Yes. TestingBot supports Appium 1, 2 and 3 against real iPhones, with full XCUITest and WebDriverAgent. Point your existing Appium suite at our hub and pick a device through capabilities. Can I test on a foldable iPhone? Yes. The iPhone Duo is available on TestingBot as an iOS 27.1 simulator for live and automated testing, before Apple releases the device. Switch between the cover screen and the inner screen during a live test to check how your layout adapts. For automated tests, set deviceName to "iPhone Duo" and version to "27.1". ## Ready to test on real iPhones? Get started with iPhone testing today. Run your tests on hosted iOS simulators or on real iPhones in the TestingBot cloud, with nothing to install and no jailbroken devices. [Start a free trial](https://testingbot.com/users/sign_up) --- URL: https://testingbot.com/integrations 36+ Integrations # One platform, every tool in your stack Plug TestingBot into the CI, project management and testing tools your team already uses. From Jenkins and GitHub Actions to Jira and Slack, 36 integrations are ready to wire up in minutes. [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) - **Integrations:** 36+ - **Categories:** 8 - **Time to first run:** \<10 min ![Selenium](https://testingbot.com/assets/integrations/selenium-1403d7bf83f64f56f05cea3fe50ae45bc3025d5a2433b7b05d2b199fec753cf6.svg) ![Jenkins](https://testingbot.com/assets/integrations/jenkins-e586a1c4af833e4c8d38d7b392a4e84bd193f30cae9cf8c9f198a58ec943a1ff.webp) ![Slack](https://testingbot.com/assets/integrations/slack-935c2233d65c4fd7ce6ac282c0abe685fcc2ed73d5d814083a071ecd4557befa.webp) ![GitHub Actions](https://testingbot.com/assets/integrations/github-actions-8436242ad9e15381f84f1a98a10dddce15611087f121d6305782e211942b7842.webp) ![Appium](https://testingbot.com/assets/integrations/appium-75e88eb18ca3b2ce63d641547ce06398c8ecd971ed17187c9134c8d205465f18.svg) ![Jira](https://testingbot.com/assets/integrations/jira-63eb4095ef566b1fe712e84f18e56b37ea706defb2b21450d93f1304c9886fa7.svg) ![Cypress](https://testingbot.com/assets/support/cypress-409c9fd3f79115ed7676aa1df4f6a217dc4402b3366a61984924aec7b4fefa97.svg) ![GitLab](https://testingbot.com/assets/integrations/gitlab-e4c0b6db07d07bc0a400f5606e258f4dc8a51b778e97bbfa51d1d531f3cb049a.webp) ![Playwright](https://testingbot.com/assets/support/playwright-2dcb4e75b55ac0d1135cfa47be6137b13f46a2047ce78b96a34f51b0e2cc0d96.svg) Trusted by some of the world's most innovative companies Most popular ## Top integrations teams set up on day one A short list our customers wire up on day one. - [![Selenium](https://testingbot.com/assets/integrations/selenium-1403d7bf83f64f56f05cea3fe50ae45bc3025d5a2433b7b05d2b199fec753cf6.svg) ](https://testingbot.com/support/web-automate/selenium) ### [Selenium](https://testingbot.com/support/web-automate/selenium) Run your Selenium tests on a cloud of 6100+ browsers. [Try with Selenium](https://testingbot.com/support/web-automate/selenium) - [![Appium](https://testingbot.com/assets/integrations/appium-75e88eb18ca3b2ce63d641547ce06398c8ecd971ed17187c9134c8d205465f18.svg) ](https://testingbot.com/support/app-automate/appium) ### [Appium](https://testingbot.com/support/app-automate/appium) Native, hybrid and web app testing on real iOS and Android. [Try with Appium](https://testingbot.com/support/app-automate/appium) - [![Jenkins](https://testingbot.com/assets/integrations/jenkins-e586a1c4af833e4c8d38d7b392a4e84bd193f30cae9cf8c9f198a58ec943a1ff.webp) ](https://testingbot.com/support/integrations/ci-cd/jenkins) ### [Jenkins](https://testingbot.com/support/integrations/ci-cd/jenkins) Drop-in pipeline plugin with tunnel and parallel support. [Try with Jenkins](https://testingbot.com/support/integrations/ci-cd/jenkins) - [![GitHub Actions](https://testingbot.com/assets/integrations/github-actions-8436242ad9e15381f84f1a98a10dddce15611087f121d6305782e211942b7842.webp) ](https://testingbot.com/support/integrations/ci-cd/github-actions) ### [GitHub Actions](https://testingbot.com/support/integrations/ci-cd/github-actions) Launch TestingBot runs from any workflow with one action. [Try with GitHub Actions](https://testingbot.com/support/integrations/ci-cd/github-actions) - [![Slack](https://testingbot.com/assets/integrations/slack-935c2233d65c4fd7ce6ac282c0abe685fcc2ed73d5d814083a071ecd4557befa.webp) ](https://testingbot.com/support/integrations/slack) ### [Slack](https://testingbot.com/support/integrations/slack) Get notified in any channel when a test finishes or fails. [Try with Slack](https://testingbot.com/support/integrations/slack) - [![Jira](https://testingbot.com/assets/integrations/jira-63eb4095ef566b1fe712e84f18e56b37ea706defb2b21450d93f1304c9886fa7.svg) ](https://testingbot.com/support/integrations/jira) ### [Jira](https://testingbot.com/support/integrations/jira) Create issues with test meta-data, screenshots and video. [Try with Jira](https://testingbot.com/support/integrations/jira) - [![GitLab](https://testingbot.com/assets/integrations/gitlab-e4c0b6db07d07bc0a400f5606e258f4dc8a51b778e97bbfa51d1d531f3cb049a.webp) ](https://testingbot.com/support/integrations/ci-cd/gitlab) ### [GitLab](https://testingbot.com/support/integrations/ci-cd/gitlab) GitLab CI integration with built-in test reporting. [Try with GitLab](https://testingbot.com/support/integrations/ci-cd/gitlab) - [![CircleCI](https://testingbot.com/assets/integrations/circleci-ede029654ff39b60342e4a5bba8fc9a3a49305cc38bda215b7cf8d07e85e0543.webp) ](https://testingbot.com/support/integrations/ci-cd/circleci) ### [CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci) CircleCI orb for one-line setup in your config.yml. [Try with CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci) ### Categories [Automation Framework 10](https://testingbot.com#automation) [Project Management 9](https://testingbot.com#project) [CI/CD 10](https://testingbot.com#ci) [Record and Playback 7](https://testingbot.com#record) [App Distribution 2](https://testingbot.com#distribution) [Browser Extension 1](https://testingbot.com#extension) [Marketplace 4](https://testingbot.com#marketplace) [Other 1](https://testingbot.com#other) [Request integration](https://testingbot.com/contact/new) ## Automation Framework Run tests with popular testing frameworks ![Selenium](https://testingbot.com/assets/integrations/selenium-1403d7bf83f64f56f05cea3fe50ae45bc3025d5a2433b7b05d2b199fec753cf6.svg) ### [Selenium](https://testingbot.com/support/web-automate/selenium) Run tests on our grid with 6100+ browsers ![Appium](https://testingbot.com/assets/integrations/appium-75e88eb18ca3b2ce63d641547ce06398c8ecd971ed17187c9134c8d205465f18.svg) ### [Appium](https://testingbot.com/support/app-automate/appium) Test native, hybrid and webapps on real devices ![Cypress](https://testingbot.com/assets/support/cypress-409c9fd3f79115ed7676aa1df4f6a217dc4402b3366a61984924aec7b4fefa97.svg) ### [Cypress](https://testingbot.com/support/web-automate/cypress) Run Cypress tests on macOS and Windows ![Puppeteer](https://testingbot.com/assets/integrations/puppeteer-7bb02294d7d9ef0b92e56f08d1bf99574298111ee79de39d298ea99efb17f322.svg) ### [Puppeteer](https://testingbot.com/support/web-automate/puppeteer) Testing with Puppeteer in the cloud ![Playwright](https://testingbot.com/assets/support/playwright-2dcb4e75b55ac0d1135cfa47be6137b13f46a2047ce78b96a34f51b0e2cc0d96.svg) ### [Playwright](https://testingbot.com/support/web-automate/playwright) Testing with Playwright in the cloud ![Espresso](https://testingbot.com/assets/integrations/espresso-a8ff30da0c385f89f4e694bb29fc4779ad132e8e3f4a777a38e608e0492ff610.svg) ### [Espresso](https://testingbot.com/support/app-automate/espresso) Mobile app testing on Android ![XCUITest](https://testingbot.com/assets/integrations/xcuitest-4a935f2368661d89d3559b3c5af3377236029478ae76bb43253f3c3bee712096.svg) ### [XCUITest](https://testingbot.com/support/app-automate/xcuitest) Automated testing on iOS ![Maestro](https://testingbot.com/assets/integrations/maestro-9d782f30f13fb4efaf9928b704c7a787eb14cb9e5b20c650137bb65892334b8b.svg) ### [Maestro](https://testingbot.com/support/app-automate/maestro) Testing on iOS and Android ![Vibium](https://testingbot.com/assets/integrations/vibium-b2e3b7b557ef98d545b87d72d96e781d23bf4f588e15f8b81d6b0d2e0727064a.png) ### [Vibium](https://testingbot.com/support/web-automate/vibium) AI-agent browser automation in the cloud ![Mobilewright](https://testingbot.com/assets/integrations/mobilewright-fdc12119ed8fb1a08f7d6116eff6fa37f5c4616acccc2b22eca478a707fd3870.png) ### [Mobilewright](https://testingbot.com/support/app-automate/mobilewright) Playwright-style mobile tests on iOS and Android ## Project Management Connect with your favorite project tools ![Slack](https://testingbot.com/assets/integrations/slack-935c2233d65c4fd7ce6ac282c0abe685fcc2ed73d5d814083a071ecd4557befa.webp) ### [Slack](https://testingbot.com/support/integrations/slack) Get notified when tests finish or fail ![WebHooks](https://testingbot.com/assets/integrations/webhooks-a941c2149b97f9fdfe154a2aad83f41206612cf0be3e61a6c77418914e32a0e6.svg) ### [WebHooks](https://testingbot.com/support/integrations/webhooks) Receive an HTTP callback on test events ![incident.io](https://testingbot.com/assets/integrations/incidentio-1993104f8073eee7edc3daa02d35b484a6d2ac259c5a51123049277ae53f05ed.svg) ### [incident.io](https://testingbot.com/support/integrations/webhooks/integrations/incident-io) Fire alerts and page on-call for failed tests ![PagerDuty](https://testingbot.com/assets/integrations/pagerduty-4f2cea281aa52eb4d414e98f7b51fa5fcf7de9d8f5f32ae149d4102aa9888de8.svg) ### [PagerDuty](https://testingbot.com/support/integrations/webhooks/integrations/pagerduty) Open incidents for failed tests ![OpsGenie](https://testingbot.com/assets/integrations/opsgenie-8b1599e3a6bfd2ce362fd52a1f01c95a6e9aebc5a62f7406f8bf71ec9b5730dc.svg) ### [OpsGenie](https://testingbot.com/support/integrations/webhooks/integrations/opsgenie) Create alerts for failed tests ![Jira](https://testingbot.com/assets/integrations/jira-63eb4095ef566b1fe712e84f18e56b37ea706defb2b21450d93f1304c9886fa7.svg) ### [Jira](https://testingbot.com/support/integrations/jira) Create issues with test meta-data ![GitHub](https://testingbot.com/assets/integrations/github-d03b3ecd2162b559b5160a1e89db53b782a287ae0402f80469c7d5be1e54646b.svg) ### [GitHub](https://testingbot.com/support/integrations/github-issue-tracker) Create issues with test meta-data ![Azure DevOps](https://testingbot.com/assets/integrations/azure-devops-f5023054e94afd0bc317b12821d90e51ab039b9cdabd7ab875a8afba875b8430.svg) ### [Azure DevOps](https://testingbot.com/support/integrations/azure-devops-issue-tracker) Create work items with test meta-data ![Trello](https://testingbot.com/assets/integrations/trello-38611a1d274409477be6bd296f5bcd470c5285b06f24ecfe71bac7dd5112c9dd.svg) ### [Trello](https://testingbot.com/support/integrations/trello-issue-tracker) Create cards from test failures ![Asana](https://testingbot.com/assets/integrations/asana-9c0452f90913962626d5b2057d62f61e969057dc2eabb1964524acf7977ab76c.svg) ### [Asana](https://testingbot.com/support/integrations/asana-issue-tracker) Create tasks from test failures ![Linear](https://testingbot.com/assets/integrations/linear-35f1036d530044281bff96db8adac28994040fe1f2a8ecf032aca7f011d26690.svg) ### [Linear](https://testingbot.com/support/integrations/linear-issue-tracker) Create issues from test failures ![Bugsnag](https://testingbot.com/assets/integrations/bugsnag-3e0b2bebfb3c599b8d986dda1bd59712ae70de4a9450b7522a698ad786ff6e62.svg) ### [Bugsnag](https://testingbot.com/support/integrations/bugsnag) Create issues with screenshots ![ZebRunner](https://testingbot.com/assets/integrations/zebrunner-51fe5d07743785e67d69f0c97a8ae327e89a01545a54662e5cf1cf3c24601bf4.svg) ### [ZebRunner](https://testingbot.com/support/integrations/zebrunner) Smart analysis of test results ## CI/CD Integrate with your continuous integration pipeline ![TeamCity](https://testingbot.com/assets/integrations/teamcity-7e75cdb835903a4983ad276c41d95e569ceac5be486e96ec1f955029ce5bb319.svg) ### [TeamCity](https://testingbot.com/support/integrations/ci-cd/teamcity) Plugin for JetBrains TeamCity ![Jenkins](https://testingbot.com/assets/integrations/jenkins-e586a1c4af833e4c8d38d7b392a4e84bd193f30cae9cf8c9f198a58ec943a1ff.webp) ### [Jenkins](https://testingbot.com/support/integrations/ci-cd/jenkins) Pipeline + tunnel support ![Bamboo](https://testingbot.com/assets/integrations/bamboo-c8fe09a3cb0b74d6055e3aa6607d77720bfe29ff487b8e3cf825138adf634346.svg) ### [Bamboo](https://testingbot.com/support/integrations/ci-cd/bamboo) Atlassian Bamboo plugin ![Azure DevOps](https://testingbot.com/assets/integrations/devops-7de50ed101385e09ac3fb0b1110a601e229282512a1b25460fdaedd22ef244bb.webp) ### [Azure DevOps](https://testingbot.com/support/integrations/ci-cd/azure) Integrate with VSTS ![CircleCI](https://testingbot.com/assets/integrations/circleci-ede029654ff39b60342e4a5bba8fc9a3a49305cc38bda215b7cf8d07e85e0543.webp) ### [CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci) Integrate with CircleCI ![Bitbucket](https://testingbot.com/assets/integrations/bitbucket-02aabcd549221726e052f2ee19c8959fbb218160ad7f7c31f6e6b96e7824a42c.webp) ### [Bitbucket](https://testingbot.com/support/integrations/ci-cd/bitbucket) Bitbucket Pipelines integration ![GitLab](https://testingbot.com/assets/integrations/gitlab-e4c0b6db07d07bc0a400f5606e258f4dc8a51b778e97bbfa51d1d531f3cb049a.webp) ### [GitLab](https://testingbot.com/support/integrations/ci-cd/gitlab) Integrate with GitLab CI ![Travis CI](https://testingbot.com/assets/integrations/travis-f3a8872deab0dd6f4c13686fd8255cb07c426309090e335a9198312b49c43232.webp) ### [Travis CI](https://testingbot.com/support/integrations/ci-cd/travis-ci) Integrate with Travis CI ![GitHub Actions](https://testingbot.com/assets/integrations/github-actions-8436242ad9e15381f84f1a98a10dddce15611087f121d6305782e211942b7842.webp) ### [GitHub Actions](https://testingbot.com/support/integrations/ci-cd/github-actions) Launch tests from workflows ![Bitrise](https://testingbot.com/assets/integrations/bitrise-d122449f8ba4609306287a7080cef27601f50d50b7bdf23811751de4d5ea41d6.svg) ### [Bitrise](https://testingbot.com/support/integrations/ci-cd/bitrise) Mobile app CI/CD integration ![Buildkite](https://testingbot.com/assets/integrations/buildkite-d573ce24dc0bb8ce8df123e751563dd8ee85b93dcdaba4a49d72ff0a678d34f6.svg) ### [Buildkite](https://testingbot.com/support/integrations/ci-cd/buildkite) Tunnel, statuses and annotations ![Gradle](https://testingbot.com/assets/integrations/gradle-a7d0123cadbbf7d2bf598be31e0b4cf9f035c9924f988603d5b8a533385ef352.svg) ### [Gradle](https://testingbot.com/support/integrations/ci-cd/gradle) Espresso tests from your build ![AWS CodePipeline](https://testingbot.com/assets/integrations/aws-37fe010e66887d20cd831130e90da33a0ec731b5dd87fbba8adc933db17b0709.webp) ### [AWS CodePipeline](https://testingbot.com/support/integrations/ci-cd/aws-codepipeline) CodeBuild buildspec with tunnel ## Record and Playback Create tests without writing code ![Selenium IDE](https://testingbot.com/assets/integrations/seleniumide-3a54afafe6c839d7e91eeb612d1bb393891df598274ecadb3557386ab2a7ab5c.webp) ### [Selenium IDE](https://testingbot.com/support/web-automate/codeless-automation/add-test) Record with browser extension ![Katalon Studio](https://testingbot.com/assets/integrations/katalon-497d6adbf9a9bb1ed8b5c9b9d6b8ded1b45dfb63a3386b572a0401515e4377ca.webp) ### [Katalon Studio](https://testingbot.com/support/web-automate/selenium/katalon-studio) Record and run on TestingBot ![Oxygen IDE](https://testingbot.com/assets/integrations/oxygen-9f738def7a7dd59f3768f44346e0fdba21ea1d9d68131c8bb474d56ccc73b1d2.webp) ### [Oxygen IDE](https://testingbot.com/support/web-automate/selenium/oxygen) Record and playback tests ![Ranorex Studio](https://testingbot.com/assets/integrations/ranorex-180a3aafc4fbb4b4d78a0d99af328c33256d85e5d7fad77c0a75a67e40ddcb69.webp) ### [Ranorex Studio](https://testingbot.com/support/integrations/ranorex) Record and run tests ![Cerberus](https://testingbot.com/assets/integrations/cerberus-21bf19476562da6ec6b57ca5acdc63bada4fa7093018801ecb2a83b9c5edb98e.webp) ### [Cerberus](https://testingbot.com/support/integrations/cerberus) Run Cerberus tests ![Tricentis Tosca](https://testingbot.com/assets/integrations/tricentis-tosca-89d025a6a71944c379e9ef745466c8b65c6af8141d72350e2bc6c33ab85b6da8.webp) ### [Tricentis Tosca](https://testingbot.com/support/integrations/tricentis-tosca) Run Tosca tests ![QMetry](https://testingbot.com/assets/integrations/qmetry-0647488fc2248e9e180e384e614b8cdbd1e8e4cf5360ab450d65a512490b0405.svg) ### [QMetry](https://testingbot.com/support/integrations/qmetry) Codeless test automation ## App Distribution Test apps from distribution platforms ![Microsoft App Center](https://testingbot.com/assets/integrations/app-center-icon-2c5c0d6c296fd04d1d3d0109db02dde20bb93a5ee7a9616e7805f27d6ec6503a.svg) ### [App Center](https://testingbot.com/support/integrations/appcenter) Test apps from Microsoft App Center ![Firebase App Distribution](https://testingbot.com/assets/integrations/firebase-a7d0366e8d92c33287523d31685c80a3472833f041aacafceb5a9a93f8a45483.svg) ### [Firebase](https://testingbot.com/support/integrations/firebase-app-distribution) Test releases from Firebase App Distribution ![Fastlane](https://testingbot.com/assets/integrations/fastlane-a14bb29fa79676049d86e42c5afd589946be86729c1493bf558240c697811fb0.png) ### [Fastlane](https://testingbot.com/support/integrations/fastlane) Upload app builds from your Fastlane lanes ## Browser Extension Test instantly from your browser ![TestingBot Extension](https://testingbot.com/assets/logo-head-2f7f08db4ac9c4a3b1a784047410f8a4ece708e7e9f10ed16e20fdd8310f8de3.svg) ### [TestingBot Extension](https://testingbot.com/support/web-live/extension) Test any webpage instantly ## Marketplace Available in popular marketplaces ![Shopify](https://testingbot.com/assets/integrations/shopify-13ec81d945cb953040bdeaf32d5267978c2d33e428ec46fc2103362cbc2cf0f7.svg) ### [Shopify](https://apps.shopify.com/testingbot-1) Test your Shopify store ![Semrush](https://testingbot.com/assets/integrations/semrush-6ea828d5e278d18118745e756c0e8359432ccbc3834045c57b6818f39d21ab92.webp) ### [Semrush](https://www.semrush.com/apps/ai-website-testing/) Test on 100+ browser combinations ![Wix](https://testingbot.com/assets/integrations/wix-1647c811fea6aa58639da859a0e269d000ae67d85beca20d18d95e8951285192.svg) ### [Wix](https://www.wix.com/app-market/web-solution/website-testing) Test your Wix website ![AWS Marketplace](https://testingbot.com/assets/integrations/aws-37fe010e66887d20cd831130e90da33a0ec731b5dd87fbba8adc933db17b0709.webp) ### [AWS Marketplace](https://aws.amazon.com/marketplace/seller-profile?id=seller-k6mxju5cqvrp4) License through AWS ## Other Additional tools and integrations ![Appium Desktop](https://testingbot.com/assets/support/appium-4b35a8bb56bae236bd84c539cee9b8bbb489a55ddf48e32d059c86685d230354.svg) ### [Appium Inspector](https://testingbot.com/support/integrations/appium-desktop) Visualize and inspect mobile app elements How it works ## From signup to first integration in under 10 minutes 1. 1 ### Grab your API key Sign up for a free account and copy your key and secret from the dashboard. 2. 2 ### Pick an integration Browse the catalog below and follow the per-tool setup guide. Most are a single config block or one CLI flag. 3. 3 ### Run your first test Trigger a build, push a commit, or click "Run" in your tool. Logs, video and screenshots stream straight back into TestingBot. Don't see your tool? ## Build a custom integration with our REST API Every integration on this page is built on the same public REST API and webhooks you can use. Drop us a note if you want us to ship a first-class plugin for your tool. [Read API docs](https://testingbot.com/support/api) [Request an integration](https://testingbot.com/contact/new) FAQ ## Common questions about integrations Is there a public API I can build my own integration against? Yes. Every integration on this page is built on the same REST API and webhook events that you can call. The full reference, including authentication, rate limits and webhook payloads, lives in the TestingBot support docs. Can you build a custom integration for our internal tool? We do build first-class integrations on request when there's broad demand. For internal tools, the public REST API plus webhooks is usually the fastest path, and most internal CI or dashboard integrations take an afternoon. Contact us with your use case and we'll point you at the right endpoints. Do the integrations cost extra? No. Every integration listed on this page is included in every paid plan and the free trial. There are no per-seat or per-integration fees. How do integrations authenticate with TestingBot? Most integrations use your TestingBot API key and secret, which you can find on the dashboard. CI plugins and the Slack/Jira/Bugsnag connectors authenticate via OAuth or signed webhooks where the target tool supports it. Will the CI/CD plugins work with self-hosted Jenkins, GitLab or Bitbucket? Yes. The plugins talk to our cloud over HTTPS and don't require an inbound connection. For tests that need to reach a private staging environment, install TestingBot Tunnel on the same network as your build agent. How do I request a new integration? Open the "Request integration" link on this page or contact support. We prioritise integrations by customer demand, so when we ship a new one, customers who asked for it are notified first. --- URL: https://testingbot.com/test-on-ipad Real iPad cloud # iPad testing online, on simulators or real iPads Run manual, automated, visual and responsive tests on iPad in the TestingBot cloud. Start on an iPadOS simulator when you want a fast first signal, then run the same tests on real iPads before you ship. iPad Pro, iPad Air, iPad mini and iPad 6th through 11th generation, with real Mobile Safari, real iPadOS apps and real multi-touch input. Nothing to install, no Mac and no jailbroken devices. [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) - **iPad models:** 6th gen → 11 - **iPadOS versions:** 12 to 27 - **Uptime SLA:** 99.9% _A real iPad running Mobile Safari with animated multi-touch gestures_ Trusted by some of the world's most innovative companies 30-second answer ## Why test on real iPads with TestingBot? iPadOS simulators and real iPads Launch a hosted iPadOS simulator in seconds for quick checks, or connect to a physical, unaltered iPad in our datacenter when you need real Apple silicon, real Mobile Safari, real multi-touch and real iPadOS. Both run from the same browser session, with no Mac and no Xcode to install. Every iPad worth testing on Pick from iPad Pro 12.9" M2, iPad Pro 11" M4, iPad Air, iPad mini and iPad 6th through 11th generation, across iPadOS 12 through iPadOS 27. Match what your customers actually use. Manual, automated, app & web Drive an iPad live from your browser, run Appium suites against the device cloud, capture screenshots for visual regression across portrait and landscape, or upload your IPA and exercise the real iPadOS app. What you can do ## Everything iPad testing needs Live debugging, automated Appium runs, mobile-web and native-app testing, multi-touch gestures, split view, network throttling and geolocation. All on a fleet of real iPads in our datacenter. - ### Live & manual testing Take an iPad in your browser. Tap, swipe, pinch, rotate and inspect, with full DevTools through Mobile Safari. Same as if you were holding the iPad in your hands. - ### Appium automation Run Appium 1, 2 and 3 suites against real iPads in parallel. WebDriverAgent, XCUITest and full XCTest support, no Mac required on your side. - ### Native iPadOS app testing Upload your IPA, install it in seconds, and exercise it on real iPads. App permissions, keyboards, push notifications and deep links all behave like real hardware. - ### Real Mobile Safari Test your website on Mobile Safari, the only browser engine Apple permits on iPadOS, on real Apple hardware. The Safari your iPad customers actually use, including the "Request Desktop Site" path. - ### Visual & screenshot testing Capture pixel-perfect screenshots across every iPad model and iPadOS version, in portrait and landscape, then diff them automatically with our visual testing service. - ### Tap, swipe, pinch & rotate Drive every multi-touch gesture exactly as a customer would, including pinch zoom, long-press, three- and four-finger swipes, split-view drag and device rotation. - ### Network throttling Test on WiFi, 5G, 4G, 3G, Edge and offline conditions. Catch slow-network bugs on cellular iPads before App Store reviewers do. - ### Portrait & landscape Rotate the iPad in one click, switch between iPad mini 8.3" and iPad Pro 12.9", and validate every breakpoint on real Mobile Safari. Device coverage ## Test On Different iPads Online iPad Pro, iPad Air, iPad mini and iPad 6th through 11th generation. Real Apple silicon, real Mobile Safari, real multi-touch input. ### iPad models - [iPad Pro 12.9" M2](https://testingbot.com/test-on-ipad/apple-ipad-pro-12-9-m2) - [iPad Pro 11" M4](https://testingbot.com/test-on-ipad/apple-ipad-pro-11-m4) - [iPad Air 13" M2](https://testingbot.com/test-on-ipad/apple-ipad-air-13-m2) - [iPad Air 11" M2](https://testingbot.com/test-on-ipad/apple-ipad-air-11-m2) - [iPad 11th gen](https://testingbot.com/test-on-ipad/apple-ipad-11) - [iPad 10th gen](https://testingbot.com/test-on-ipad/apple-ipad-10) - [iPad 9th gen](https://testingbot.com/test-on-ipad/apple-ipad-9) - [iPad 8th gen](https://testingbot.com/test-on-ipad/apple-ipad-8) - [iPad 7th gen](https://testingbot.com/test-on-ipad/apple-ipad-7) - [iPad 6th gen](https://testingbot.com/test-on-ipad/apple-ipad-6) - [iPad mini 7](https://testingbot.com/test-on-ipad/apple-ipad-mini-7) - [iPad mini 6](https://testingbot.com/test-on-ipad/apple-ipad-mini-6) [View all iPads and iPhones](https://testingbot.com/support/app-automate/devices) ![iPadOS](https://testingbot.com/assets/environments/svg/ios-383468addf160fa18d0e431f529420739d7f7d1206175f682fead627d2e99a52.svg) ### iPadOS versions - iPadOS 27 - iPadOS 26 - iPadOS 18 - iPadOS 17 - iPadOS 16 - iPadOS 15 - iPadOS 14 - iPadOS 13 - iPadOS 12 Latest iPadOS releases land on the cloud within hours of Apple shipping them. Older majors stay around for legacy regression coverage. iPadOS simulator vs real iPad ## When an iPad simulator is enough, and when you need a real iPad TestingBot gives you both. An iPadOS simulator is the right tool for a fast pass on layout, navigation and obvious breakage, and it is cheap to run on every commit. A real iPad is what you want before release, because a simulator cannot reproduce Apple silicon performance, the shipped WebKit, biometrics or real multi-touch. Here is where the two differ. | Capability | TestingBot, real iPads | iPadOS simulator | | --- | --- | --- | | Hardware | Real Apple silicon, real iPad in our datacenter | x86 or ARM Mac, no real Apple radios or sensors | | Mobile Safari | Real WebKit shipped on the device by Apple | Simulator WebKit, drifts from shipped Safari | | Multi-touch & gestures | Real multi-touch, split view, drag-and-drop, rotation | Mouse-faked taps, no real multi-touch | | Native apps | Real iPadOS, real StoreKit, ClassKit and HealthKit | Some Apple frameworks missing or stubbed | | Device coverage | iPad Pro, Air, mini and iPad 6th to 11th generation | Hosted iPadOS simulator images, no Mac or Xcode install | | Video, logs and HAR | HD video, device logs, HAR and screenshots | Roll your own recording and capture | Native iPadOS apps ## Native iPadOS app testing Upload your IPA and TestingBot installs it on a real iPad in seconds. Drive the app live or via Appium, capture full HD video, syslog and crash reports, and exercise push notifications, deep links and the iPadOS keyboard exactly as your customers will. [Read more](https://testingbot.com/features/automation/appium) upload-and-test.js // 1) Upload your IPA to TestingBot storageconst TestingBot = require('testingbot-api'); const tb = new TestingBot({ api\_key: 'YOUR\_KEY', api\_secret: 'YOUR\_SECRET' }); tb.uploadFile('./MyApp.ipa', (err, res) =\> { // res.app\_url → "tb://..."// 2) Run your Appium test on a real iPad driver.init({ platformName: 'iOS', 'appium:deviceName': 'iPad 9th generation', 'appium:platformVersion': '27', 'appium:realDevice': true, 'appium:app': res.app\_url }); }); Live exit nodes 10+ countries United States United Kingdom Germany France Spain Belgium Australia China Japan Geolocation testing ## Geolocation Testing on iPads Run iPad tests from real exit nodes around the world to verify geo-targeting, geo-blocking and localisation from the United States, United Kingdom, Germany, France, Spain, Belgium, Australia, China, Japan and more. Secure Tunnel ## Test internal & staging sites on real iPads Use our [secure tunnel](https://testingbot.com/support/tunnel) to run iPad tests against firewalled staging sites, internal dashboards or apps still on localhost. Encrypted end-to-end, no inbound firewall rules required, ready out of the box for popular CI providers. ![TestingBot secure tunnel for iPad](https://testingbot.com/assets/features/tunnel-diagram-poster-8c25e19e6fe4f7276a46a96b31a0ad0e07bb4f5bf98dc192d2b70897cfbb1a26.png) [Read more](https://testingbot.com/support/tunnel) Secure devices ## Real iPads. Wiped after every session. Each session locks a single iPad for your test only. The moment your test ends, the device is wiped, all installed apps are deleted, caches are cleared, and the iPad is returned to the pool ready for the next customer. GDPR-compliant, with SAML SSO and audit logs available for enterprise teams. - One real iPad per session, never shared - Wiped between every session - Encrypted IPA uploads with per-app retention controls 1 customer per session Datacenter hosted Wiped after each test CSA STAR Level 1 GDPR compliant SAML SSO enterprise Test without a Mac ## Test on iPads from any OS, no Mac required On Windows or Linux and need to validate something on an iPad? You don't have to buy a Mac, set up Xcode, or chase down a borrowed iPad. TestingBot streams a real iPad from our datacenter directly to any browser, so you can drive a physical iPad Pro, iPad Air, iPad mini or any older generation from Windows 11, 10, Linux or any PC. The streamed session is the real device, with Mobile Safari, the iPadOS keyboard, gestures and notifications. You drive it from any browser on your local machine, without installing Xcode or owning Apple hardware. [Start free](https://testingbot.com/users/sign_up) No Mac to buy No Xcode to install Real iPad in your browser FAQ ## Frequently Asked Questions Which iPads and iPadOS versions are supported? TestingBot supports physical iPads from iPad 6th generation through the latest iPad Pro and iPad Air models, including the iPad mini line. iPadOS versions range from iPadOS 12 through iPadOS 27. New iPads are added on or close to the day Apple ships them. How do I test a mobile app on iPad? Testing your iPadOS app on a variety of iPad devices is important before publishing to the App Store. TestingBot allows you to upload and test your app on a number of different iPads. Choose between various iPadOS versions and screen sizes, to make sure your mobile app works on every iPad your customers might own. Why is testing your website on an iPad necessary? iPads have a meaningful share of global tablet usage, and on tablets your customers use Mobile Safari. It is important to make sure your website looks and behaves correctly on iPad. Safari is the default browser on iPad, so you should test your website on Mobile Safari to make sure iPad users have a good experience. Test in portrait and landscape mode to make sure your website and apps work correctly on tablets. Should I use an iPadOS simulator or a real iPad? Use both, at different points. An iPadOS simulator is fast to launch and cheap to run on every commit, and it is good for catching layout breakage, broken navigation and obvious regressions. TestingBot hosts iPadOS simulators so you can do that without a Mac or an Xcode install. Before you ship, move to a real iPad. A simulator does not mimic exact device performance, several Apple frameworks (StoreKit, ClassKit, biometrics, push) are missing or stubbed, and because it cannot reproduce real hardware, battery, GPU and sensor measurements are not accurate. The practical split is simulators for speed during development, real devices for release confidence. Can I test Safari on iPad? TestingBot provides access to unaltered iPads. These come straight from the store and are not jailbroken. Run your tests on real Mobile Safari, on physical iPad devices in our datacenter. Can I test on iPad without owning a Mac or iPad? Yes. TestingBot streams real iPads to any Windows, macOS or Linux browser. You get the real device, real Mobile Safari and real iPadOS, without owning a Mac, an iPad or a copy of Xcode. Can I run Appium against your iPad cloud? Yes. TestingBot supports Appium 1, 2 and 3 against real iPads, with full XCUITest and WebDriverAgent. Point your existing Appium suite at our hub and pick an iPad model and iPadOS version through capabilities. ## Ready to test on real iPads? Get started with iPad testing today. Run your tests on hosted iPadOS simulators or on real iPads in the TestingBot cloud, with nothing to install and no jailbroken devices. [Start a free trial](https://testingbot.com/users/sign_up) --- URL: https://testingbot.com/test-on-galaxy Real Samsung Galaxy cloud # Samsung Galaxy testing online, on emulators or real phones Run manual, automated, visual and responsive tests on Samsung Galaxy in the TestingBot cloud. Start on an Android emulator when you want a fast first signal, then run the same tests on real Galaxy hardware before you ship. From Galaxy S8 through the new Galaxy S26, plus Galaxy A and Galaxy Tab, with real Chrome on Android, real apps and real touch input. Nothing to install, no rooted devices. [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) - **Galaxy models:** S8 to S26 - **Android versions:** 9 to 16 - **Uptime SLA:** 99.9% _A real Samsung Galaxy running Chrome for Android with animated tap gestures_ Trusted by some of the world's most innovative companies 30-second answer ## Why test on real Samsung Galaxy with TestingBot? Android emulators and real Samsung devices Run on a hosted Android emulator in seconds for quick checks, or connect to a physical, unmodified Samsung Galaxy phone or tablet in our datacenter when you need real Samsung hardware, real Chrome on Android and real touch and sensors. Both run from the same browser session, with no local Android SDK to install. Every Galaxy worth testing on Pick from Galaxy S26, S25, S24, S23, S21, S20, S10 and S8, plus Galaxy A55, A12 and Galaxy Tab A, across Android 9 through Android 16 and Samsung One UI. Match what your customers actually carry. Manual, automated, app & web Drive a Galaxy live from your browser, run Appium, Espresso or UiAutomator2 suites against the device cloud, capture screenshots for visual regression, or upload your APK and exercise the real Android app. What you can do ## Everything Samsung Galaxy testing needs Live debugging, automated Appium and Espresso runs, mobile-web and native-app testing, gestures, network throttling, geolocation. All on a fleet of real Samsung Galaxy devices in our datacenter. - ### Live & manual testing Take a real Galaxy in your browser. Tap, swipe, type, rotate and inspect, with full Chrome DevTools through the Android browser. Same as if you were holding the phone in your hand. - ### Appium automation Run Appium 1, 2 and 3 suites against real Galaxy phones and tablets in parallel. UiAutomator2 driver, full XPath and accessibility locators, no signing certificates to wrangle. - ### Espresso & UiAutomator2 Upload your APK plus instrumentation APK and TestingBot runs Espresso or UiAutomator2 on a real Galaxy from any CI runner, no local Android SDK required. - ### Native Android app testing Upload your APK or AAB, install it in seconds, and exercise it on real Galaxy devices. Permissions, biometrics, keyboards, push notifications and deep links all behave like real hardware. - ### Real Chrome on Android Test your website on Chrome for Android, the browser most of your mobile users actually use, on a real Samsung Galaxy with real Android WebView behavior. - ### Visual & screenshot testing Capture pixel-perfect screenshots across every Galaxy model and Android version, then diff them automatically with our visual testing service. - ### Tap, swipe, pinch & rotate Drive every multi-touch gesture exactly as a customer would, including pinch zoom, long-press, scroll, three-finger swipes and device rotation. - ### Network throttling Test on WiFi, 5G, 4G, 3G, Edge and offline conditions. Catch slow-network bugs before Play Store reviewers do. - ### Phones & tablets Switch between portrait and landscape, swap between a flagship Galaxy S26 and a Galaxy Tab A in one click, and validate every breakpoint on real Chrome for Android. Device coverage ## Test On Galaxy Devices Online Galaxy S26, S25, S24, S23, S21, S20, S10, S8, plus Galaxy A55, A12 and Galaxy Tab A. Real Samsung hardware, real Chrome on Android, real touch input. ### Galaxy models - Galaxy S26 - Galaxy S25 - Galaxy S24 - Galaxy S23 - Galaxy S21 - Galaxy S20 - Galaxy S10 - Galaxy S8 - Galaxy A55 - Galaxy A12 - Galaxy Tab A [View all Galaxy devices](https://testingbot.com/support/app-automate/devices) ### Android versions - Android 16 - Android 15 - Android 14 - Android 13 - Android 11 - Android 10 - Android 9 Latest Android releases land on the cloud within hours of Samsung shipping the One UI update. Older majors stay around for legacy regression coverage. Android emulator vs real Galaxy ## When an Android emulator is enough, and when you need a real Galaxy TestingBot gives you both. An Android emulator is the right tool for a fast first pass on layout, navigation and obvious breakage, and it is cheap to run on every commit. A real Galaxy is what you want before release, because an emulator cannot reproduce Samsung hardware, One UI behaviour, real radios or real touch. Here is where the two differ. | Capability | Real Galaxy devices | Android emulator | | --- | --- | --- | | Hardware | Real Samsung Galaxy phone in our datacenter | x86 host CPU, no real Samsung radios or sensors | | Chrome on Android | Real Chrome and Android WebView shipped on the device | Emulator Chrome, drifts from shipped Android Chrome | | Touch & gestures | Real multi-touch, real rotation, real haptics | Mouse-faked taps, no haptics | | Native apps | Real Android, real Play Services, real Samsung One UI | Generic AOSP, missing or stubbed Samsung-specific behaviour | | Device coverage | Galaxy S8 through Galaxy S26, plus Galaxy A and Galaxy Tab A | Hosted Android system images, no local SDK or download | | Video, logs and HAR | HD video, logcat, HAR and screenshots | Roll your own recording and capture | Native Android apps ## Native Android app testing Upload your APK or AAB and TestingBot installs it on a real Galaxy in seconds. Drive the app live or via Appium, Espresso or UiAutomator2, capture full HD video, logcat and crash reports, and exercise biometrics, push notifications, deep links and the Samsung keyboard exactly as your customers will. Every Galaxy above is physical hardware racked in our European datacenter, not an emulator. The [real device testing](https://testingbot.com/real-device-testing) overview lists the full fleet, explains how each handset is wiped and re-imaged between sessions, and shows which plans include it. [Read more](https://testingbot.com/features/automation/appium) upload-and-test.js // 1) Upload your APK to TestingBot storageconst TestingBot = require('testingbot-api'); const tb = new TestingBot({ api\_key: 'YOUR\_KEY', api\_secret: 'YOUR\_SECRET' }); tb.uploadFile('./MyApp.apk', (err, res) =\> { // res.app\_url → "tb://..."// 2) Run your Appium test on a real Galaxy driver.init({ platformName: 'Android', 'appium:deviceName': 'Samsung Galaxy S26', 'appium:platformVersion': '16', 'appium:realDevice': true, 'appium:app': res.app\_url }); }); Live exit nodes 10+ countries United States United Kingdom Germany France Spain Belgium Australia China Japan Geolocation testing ## Geolocation Testing on Galaxy Run Galaxy tests from real exit nodes around the world to verify geo-targeting, geo-blocking and localisation from the United States, United Kingdom, Germany, France, Spain, Belgium, Australia, China, Japan and more. Secure Tunnel ## Test internal & staging sites on real Galaxy devices Use our [secure tunnel](https://testingbot.com/support/tunnel) to run Galaxy tests against firewalled staging sites, internal dashboards or apps still on localhost. Encrypted end-to-end, no inbound firewall rules required, ready out of the box for popular CI providers. ![TestingBot secure tunnel for Galaxy](https://testingbot.com/assets/features/tunnel-diagram-poster-8c25e19e6fe4f7276a46a96b31a0ad0e07bb4f5bf98dc192d2b70897cfbb1a26.png) [Read more](https://testingbot.com/support/tunnel) Secure devices ## Real Galaxy devices. Wiped after every session. Each session locks a single Samsung Galaxy for your test only. The moment your test ends, the device is wiped, all installed apps are deleted, caches are cleared, and the device is returned to the pool ready for the next customer. GDPR-compliant, with SAML SSO and audit logs available for enterprise teams. - One real Samsung Galaxy per session, never shared - Wiped between every session - Encrypted APK uploads with per-app retention controls 1 customer per session Datacenter hosted Wiped after each test CSA STAR Level 1 GDPR compliant SAML SSO enterprise Test from any OS ## Test on Galaxy from any OS, no Android device required On Windows, macOS or Linux and need to validate something on a Samsung Galaxy? You don't have to buy the hardware, set up Android Studio, or chase down a borrowed device. TestingBot streams a real Galaxy from our datacenter directly to any browser. The streamed session is the real device, with Chrome for Android, the Samsung keyboard, biometrics and notifications. You drive it from any browser on your local machine, without installing Android Studio or owning a Samsung phone. Not tied to a specific handset? Run a generic [Android emulator online](https://testingbot.com/android-emulator-online) instead. [Start free](https://testingbot.com/users/sign_up) No device to buy No SDK to install Real Galaxy in your browser FAQ ## Frequently Asked Questions Which Samsung Galaxy models and Android versions are supported? TestingBot supports physical Samsung Galaxy phones from Galaxy S8 through the new Galaxy S26, plus Galaxy A55, A12 and Galaxy Tab A. Android versions range from Android 9 through Android 16, and we keep Samsung One UI majors current. New flagship Galaxy phones are added on or close to the day Samsung ships them. Why should I test on Samsung Galaxy devices? Samsung Galaxy devices have roughly 28% global mobile market share, and meaningfully higher in markets like Brazil, India and South Korea. Your Android coverage is incomplete without them. TestingBot lets you test on the most popular Galaxy phones and tablets, with real Samsung One UI behaviour and the same Chrome for Android your users are running. Why use TestingBot to test on Samsung Galaxy? TestingBot provides physical Samsung devices, ranging from Galaxy S8 through Galaxy S26, plus Galaxy A55, A12 and Galaxy Tab A, hosted in our EU datacenter. You instantly interact with a remote, physical device, including full Chrome DevTools, Android logcat and a downloadable video of the session. Should I use a Samsung Galaxy emulator or a real device? Use both, at different points. An Android emulator is fast, cheap to run on every commit and perfectly good for catching layout breakage, broken navigation and obvious regressions. TestingBot hosts Android emulators so you can do that without installing the Android SDK locally. Before you ship, move to a real Galaxy. An emulator runs a generic AOSP system image on host CPU, so it misses biometrics, Samsung One UI behaviour, push notifications, real-network conditions and the exact Chrome for Android build that Samsung ships. Google also recommends testing on real Android devices before submitting to the Play Store. The practical split is emulators for speed during development, real devices for release confidence. Can I test Chrome on Android with Samsung Galaxy? Yes. Every Samsung Galaxy device in the TestingBot cloud ships with Chrome for Android and the Samsung Internet browser, both unmodified and up-to-date. Devices are not rooted and come with Google Play Services and Chrome for Android, just like a retail device. Run Mobile-Web tests on real Chrome, not on a desktop browser pretending to be one. Can I run Appium and Espresso against your Galaxy cloud? Yes. TestingBot supports Appium 1, 2 and 3 with the UiAutomator2 driver, plus native Espresso runs. Point your existing Android suite at our hub and pick a Galaxy model and Android version through capabilities. Can I test on Galaxy without owning the device? Yes. TestingBot streams real Galaxy phones to any Windows, macOS or Linux browser. You get the real device, real Chrome for Android and real Samsung One UI, without buying the hardware or installing Android Studio. ## Ready to test on real Samsung Galaxy? Get started with Galaxy testing today. Run your tests on hosted Android emulators or on real Samsung devices in the TestingBot cloud, with nothing to install and no rooted devices. [Start a free trial](https://testingbot.com/users/sign_up) --- URL: https://testingbot.com/android-emulator-online Android emulator in your browser # Android emulator online, no SDK required Run a virtual Android phone on our servers and drive it from your browser. Upload an APK or open any URL in Chrome on Android, with no SDK to install and no local hardware. Real Android devices sit beside them for release validation. [Start free](https://testingbot.com/users/sign_up) [See plans](https://testingbot.com/pricing) 100 free minutes to start. No credit card, no Android SDK, nothing to install. - **Emulator configurations:** 53 - **Device models:** 31 - **Android versions:** 4.4–16 android-emulator.js // Start a browser session on an Android emulator driver.init({ platformName: 'Android', 'appium:deviceName': 'Pixel 8', 'appium:platformVersion': '16', 'appium:realDevice': false, 'appium:browserName': 'Chrome' }); // Same test, real hardware: realDevice → true One capability switches an emulator run onto physical hardware. Drive it with [Appium](https://testingbot.com/support/app-automate/appium), or open a session and tap through it by hand. ## What is an Android emulator online? An online Android emulator is a virtual Android device that runs on a remote server and streams to your web browser, so you can install an APK or open a site in Chrome on Android without an Android SDK, a system image or local hardware. It behaves like a phone in software: the operating system, the browser and your app are real, the hardware underneath is not. In a browser session ## What you can do ### Install an APK Upload a build to TestingBot Storage and install it on an emulator session, by hand or from a test. ### Open any site in Chrome on Android Load a public URL, or reach a staging environment behind your firewall through TestingBot Tunnel. ### Capture screenshots and video Every session records video and logs, so a failure comes with evidence rather than a stack trace alone. ### Automate it Drive the same session with Appium or Espresso instead of tapping through it by hand. Framework guides: [Appium](https://testingbot.com/support/app-automate/appium) and [Espresso](https://testingbot.com/support/app-automate/espresso). Scale and CI ## Run emulators in parallel from your pipeline One emulator in a browser tab is useful for a quick look. A suite needs many at once. Emulator sessions draw on the same concurrency as every other session, so the same plan that runs your browser tests runs these, and [running them in parallel](https://testingbot.com/features/parallel-testing) is what turns a long suite into a short one. Wire it into [Jenkins](https://testingbot.com/support/integrations/ci-cd/jenkins), [GitHub Actions](https://testingbot.com/support/integrations/ci-cd/github-actions), [CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci), [Bitrise](https://testingbot.com/support/integrations/ci-cd/bitrise) or [GitLab](https://testingbot.com/support/integrations/ci-cd/gitlab) and emulator runs happen on every push. The [full CI/CD integration list](https://testingbot.com/support/integrations/ci-cd) covers the rest. Where emulators stop ## Emulators for speed, real devices for release confidence An emulator is a software model of a phone running on server hardware. It boots in seconds, costs less per minute and scales further, which is exactly what you want on every commit. What it cannot reproduce is physical hardware: the real GPU, the camera, biometric sensors, battery behaviour and genuine network conditions. So we do not claim emulators replace hardware. TestingBot hosts 53 emulator configurations and 35 real Android devices across 22 models, and most teams use both: emulators in CI on every push, [real Android devices](https://testingbot.com/real-device-testing) on the release branch. | Use it for | Emulator | Real device | | --- | --- | --- | | Functional flows and regression in CI | Yes | Yes | | Responsive web testing in Chrome | Yes | Yes | | High-volume parallel runs on every commit | Yes | Works, but slower | | GPU rendering, camera and sensors | Use hardware | Yes | | Biometrics, push notifications, battery | Use hardware | Yes | | Final release validation | Use hardware | Yes | Coverage ## Android versions available Emulator images span Android 4.4 to 16, so you can reproduce a bug on an old release and verify the fix on the newest one in the same suite. Android 4.4 Android 5 Android 6 Android 7.1 Android 8 Android 9 Android 10 Android 11 Android 12 Android 13 Android 14 Android 15 Android 16 Looking for a specific handset rather than a generic Android image? [Samsung Galaxy testing](https://testingbot.com/test-on-galaxy) covers the Samsung Galaxy range on physical hardware. ## Getting started 1. 1 Create a TestingBot account. The free plan needs no credit card. 2. 2 Pick an Android version and device model, and start a session in your browser. 3. 3 Upload your APK, or type a URL to open it in Chrome on Android. 4. 4 When you are ready to automate, point Appium at the same capabilities. ## Frequently asked questions - **Can I run an Android emulator online without installing anything?:** Yes. TestingBot runs the emulator in our datacenter and streams it to your browser, so there is no Android SDK, no system image download and no local hardware requirement. You open a session, interact with it in the browser, and it is wiped when you finish. - **Can I run an APK on an online Android emulator?:** Yes. Upload your .apk to TestingBot Storage and install it on an emulator session, manually or from an automated test. You can also drive it with Appium or Espresso rather than tapping through it by hand. - **What is the difference between an Android emulator and a real Android device?:** An emulator is a software model of a phone running on server hardware. It boots faster, costs less and scales further, which makes it a good fit for functional checks and CI. A real device is physical hardware with a real GPU, real sensors, a real camera and real battery behaviour. Use emulators for speed and real devices for release confidence. - **Which Android versions can I emulate?:** TestingBot hosts emulator images spanning Android 4.4 to 16, across 31 device models, so you can reproduce a bug on an old release and verify the fix on the newest one in the same suite. - **Can I test a website rather than an app?:** Yes. Every emulator session runs Chrome on Android, so you can load any public URL, or reach a staging site behind your firewall through TestingBot Tunnel. Automated web tests run with Selenium, Playwright or Appium against the same session. - **Is an online Android emulator good enough for release testing?:** For most functional coverage, yes. For anything touching the GPU, camera, biometrics, push notifications, real network conditions or battery, test on physical hardware before you ship. Most teams run emulators on every commit and real devices on the release branch. --- URL: https://testingbot.com/support Developer platform # Documentation TestingBot is a cloud testing platform with 6100+ real browsers and physical iOS and Android devices. Use this guide to install your test framework, run your first test, then scale to CI. ## Explore by testing type ### [Automated Web Testing](https://testingbot.com/support/web-automate/selenium/nodejs) Get started with automated testing of websites. - [Selenium](https://testingbot.com/support/web-automate/selenium) - [Cypress](https://testingbot.com/support/web-automate/cypress) - [Puppeteer](https://testingbot.com/support/web-automate/puppeteer) - [Playwright](https://testingbot.com/support/web-automate/playwright) [Browse all 6100+ remote browsers →](https://testingbot.com/support/web-automate/browsers) ### [Automated App Testing](https://testingbot.com/support/app-automate) Run automated tests against native mobile apps. - [Appium](https://testingbot.com/support/app-automate/appium) - [Espresso](https://testingbot.com/support/app-automate/espresso) - [XCUITest](https://testingbot.com/support/app-automate/xcuitest) - [Maestro](https://testingbot.com/support/app-automate/maestro) [Browse all physical iOS and Android devices →](https://testingbot.com/support/app-automate/devices) ### [AI Testing](https://testingbot.com/support/ai) Use AI to test. Connect remote browsers with LLM frameworks. - [Codeless tests with AI](https://testingbot.com/support/ai/codeless) - [Generate tests with LLM](https://testingbot.com/support/ai/chat) - [Test Recorder](https://testingbot.com/support/web-automate/codeless-automation/add-test) - [Functions](https://testingbot.com/support/functions) [AI Integrations →](https://testingbot.com/support/ai/integrations) ### [Manual Web & App Testing](https://testingbot.com/support/web-live) Control a remote browser or device interactively. - [Manual Web Testing](https://testingbot.com/support/web-live) - [Manual App Testing](https://testingbot.com/support/app-live) [About live testing →](https://testingbot.com/features/manual-browser-testing) ### [Visual Website Testing](https://testingbot.com/support/visual-testing) Capture and compare screenshots across browsers. - [Automated Visual Testing](https://testingbot.com/support/visual-testing/automated) - [Codeless Visual Testing](https://testingbot.com/support/visual-testing/codeless) [About visual testing →](https://testingbot.com/features/screenshot-testing) ### [Accessibility Web Testing](https://testingbot.com/support/accessibility) Test your URLs against WCAG accessibility rules. - [Selenium + Accessibility](https://testingbot.com/support/accessibility/web/selenium) - [Scheduled Testing](https://testingbot.com/support/accessibility/web/scheduled) [About accessibility testing →](https://testingbot.com/features/accessibility-testing) ## What's new ### [Latest from the blog](https://testingbot.com/resources/articles) Tutorials, deep dives and release notes for TestingBot. - [Parallel Testing: What Actually Scales](https://testingbot.com/resources/articles/parallel-testing) Sep 1, 2026 - [Automated Regression Testing in CI](https://testingbot.com/resources/articles/automated-regression-testing) Aug 31, 2026 - [Mobile App Test Automation at Scale](https://testingbot.com/resources/articles/mobile-app-test-automation) Aug 31, 2026 - [AI in Software Testing: What Works](https://testingbot.com/resources/articles/ai-in-software-testing) Aug 31, 2026 [See all product updates](https://testingbot.com/product-updates) ### Platform info [All systems operational](https://status.testingbot.com) Live service status View → [Trust Center](https://trust.testingbot.com) Compliance - GDPR - CSA STAR - EU hosted Resources - [Changelog Latest releases](https://testingbot.com/product-updates) - [REST API Endpoint reference](https://testingbot.com/support/api) - [Configuration Capabilities reference](https://testingbot.com/support/other/configuration) - [FAQ Common questions](https://testingbot.com/support/faq) - [llms.txt Sitemap for AI agents](https://testingbot.com/llms.txt) ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/faq # Frequently asked questions Answers to the most common questions about running Selenium, Appium and cloud testing on TestingBot. For framework-specific guides see the [documentation index](https://testingbot.com/support); for billing and account questions contact [info@testingbot.com](mailto:info@testingbot.com). ## Questions How does TestingBot's cloud test grid work? TestingBot runs an open-source Selenium and Appium grid that has been tuned for speed and reliability. Every test runs on a freshly-provisioned virtual machine (or real iOS/Android device). When the test ends, the VM is destroyed so there is no state carried between runs. You connect to the grid by changing your WebDriver endpoint to https://hub.testingbot.com/wd/hub and passing your API key and secret as capabilities — no other changes to your test code are required. How are costs calculated and how much will I be charged? TestingBot accounts include a pool of free minutes you can use to evaluate the service. After that, billing is based on test minutes and the number of parallel tests in your plan. We charge from the moment the test session starts until it ends — queue time and failures caused on our side are not billed. See [pricing](https://testingbot.com/pricing) for current plans, and [contact us](https://testingbot.com/contact/new) for volume or enterprise pricing. What happens if my test does not seem to run on your system? If a test does not start or behaves unexpectedly, check the dashboard at https://testingbot.com/members for an error code on the session. Common causes are an invalid browser/OS combination, missing or expired API credentials, or a tunnel that did not connect. Errors that originate on TestingBot's infrastructure are logged on our side and are not billed. Contact info@testingbot.com or open Slack with the session id and we will investigate. Why don't I see screenshots after my test finished? Screenshots and video are uploaded to our storage after the test completes and can take up to a few minutes to appear in the dashboard depending on the length of the session. If screenshots are still missing after 5 minutes, confirm that you have not disabled them via the screenrecorder or screenshots capabilities, and check that your session ended with driver.quit() (or the equivalent) rather than being killed mid-flight. My test fails with "could not find browser/operating system" — what's wrong? A CapabilityNotPresentOnTheGridException (or a similar "browser not found" error) means the browser + version + platform combination you requested is not currently offered. Check the live list of [supported browser combinations](https://testingbot.com/support/web-automate/browsers) and adjust your capabilities. Use latest, latest-1, latest-2 etc. for the major-version aliases if you do not need to pin to an exact version. Can I run my tests in parallel on TestingBot? Yes. Every plan includes a number of concurrent test sessions — you can run that many tests at the same time. Each session is fully isolated on its own VM or device, so parallel tests never interfere with each other. Most CI test runners (pytest-xdist, Jest, Mocha-parallel, Maven Surefire) and Selenium Grid clients launch parallel sessions natively. How long are test artifacts (logs, screenshots, video) kept? Test session metadata, screenshots, video recordings and logs are retained for 30 days. You can also delete a session manually from the dashboard or via the REST API at any time, and immediate deletion after each test can be configured. Test data is stored in our EU data centres; see the [Trust Center](https://trust.testingbot.com) for compliance details. Does TestingBot integrate with CI/CD systems like GitHub Actions, Jenkins or CircleCI? Yes. TestingBot integrates with all major CI/CD systems — GitHub Actions, Jenkins, CircleCI, GitLab CI, Bitbucket Pipelines, Travis CI, TeamCity and Azure DevOps among others. You point your existing Selenium or Appium test runner at the TestingBot endpoint and add your key and secret as environment variables. See [integration guides](https://testingbot.com/support/integrations) for CI templates and recipes. How secure is TestingBot? Do you support SSO and tunnels? TestingBot is GDPR-compliant and registered in the CSA STAR registry (Level 1 self-assessment); our CAIQ, Technical & Organizational Measures and Bitsight rating are published in the [Trust Center](https://trust.testingbot.com). The infrastructure is hosted in the EU. SAML 2.0 single sign-on (Okta, Azure AD, Google Workspace) is available on Enterprise plans. To test pre-production sites, use the [TestingBot Tunnel](https://testingbot.com/support/tunnel), an encrypted connection between your network and the test VM. What's the difference between Live (manual) and Automated testing on TestingBot? Automated testing runs Selenium, Appium, Cypress, Playwright, Puppeteer or other framework scripts against the grid programmatically. Live testing gives you an interactive remote browser or device window — you point and click yourself, exactly like using a local browser. Most teams use both: automation for regression suites and live testing for exploratory testing and reproducing bugs reported by users. Pricing for each is separate (see [pricing](https://testingbot.com/pricing)). ## Need more help? - Browse the full [documentation](https://testingbot.com/support) for framework-specific guides. - Join the [TestingBot Slack workspace](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw). - Email [info@testingbot.com](mailto:info@testingbot.com) with your session id for fastest help. - Check the [status page](https://status.testingbot.com) for known incidents. --- URL: https://testingbot.com/support/analytics # Analytics TestingBot provides a suite of graphs and data tables to help you understand and improve the quality of your tests. - [Guide](https://testingbot.com/support/analytics/guide) - [Test Insights](https://testingbot.com/support/analytics/insights) - [Build Analytics](https://testingbot.com/support/analytics/builds) ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/analytics/insights # Test Insights The Test Insights page provides, for each test you run on TestingBot, a scatter-plot visualization. This is a great way to visualize how your test is doing with regards to **performance** and **flakiness** over time. You'll get a visual overview of how the test has been running the previous weeks and months. Discover trends and outliers and find out how robust your test has run over time. The only prerequisite is that you've [annotated your webdriver sessions with the same name](https://testingbot.com/support/api/tests#updatetest). Only then can we track how your test has been performing over time. ## Viewing Test Insights for a test In our [member dashboard](https://testingbot.com/members), navigate to the test details page of a test session. Click the `test name` on the `test details page` to open the Test Insights page for this test. **Important:** Only tests that have a name (and not `unnamed test...`) will show this option. ## Reading the Scatter Plot Visualization On this page you'll find a scatter plot with some additional metrics on top of the graph. Performance metrics include: - **Total Runs:** how many times you've run this test in the past. - **Total Failures:** how many times in the past did this test fail. - **Fastest Runtime:** how many seconds did the fastest run of this test take in the past. - **Slowest Runtime:** how many seconds was the slowest execution of this test in the past. ![Test Scatter Plot](https://testingbot.com/assets/support/analytics/insights-db60d9fc6ca0cc9c4f0c7294d337547c1db38786b4036fa302faf486f4cf6131.png) The example graph above shows what to expect from the Test Insights page. This is a scatter plot, where each dot represents a session with the identical name. The `X-axis` indicates a time range (which you can specify in the filter below the graph). The `Y-axis` indicates the duration (in seconds) of each test. The color of the dot indicates if the test passed (green), failed (red) or unknown (gray). TestingBot does not know your test status by default, you need to [annotate your tests with a success parameter](https://testingbot.com/support/api/tests#updatetest). ## Interpreting results You can pan the graph to zoom in or out, and use the filter below the graph to change the time range. Clicking a dot will navigate to the test details page of that test session. The example above shows a very consistent graph, with a high pass rate and a couple of (failed) outliers. Other graphs may show a different story. For example, a flaky test will show a scatter plot with `failures` on first run and `passes` on re-runs. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/analytics/builds # Test Builds Test Builds are group of tests. This term is usually used in a CI/CD context. Whenever you want to deploy to production, you'll run a group of tests to verify the build. It is possible to [annotate your Selenium WebDriver tests with a build identifier](https://testingbot.com/support/web-automate/selenium/test-options#build). When TestingBot sees a `build identifier` in the desired capabilities of your test, we will add the test to the group you specified. ## Build Graph In our [member dashboard](https://testingbot.com/members), navigate to the build overview page and click a build name. You will see a page with a pie-chart of the build's success rate and a table with all tests included in this build. ![Build Analytics](https://testingbot.com/assets/support/analytics/build-d16f497f14b60b08e5a47b0ab2b59b4080a905946fd5e95fc52b6131955b812b.png) Click the `green` or `red` slice of the pie-chart to see a list of `passed` or `failed` tests. ## Build Efficiency For each build, we calculate an `efficiency percentage`. This number indicates how well your test runner has been using `parallelization` to shorten the total test duration of the test build. One of the advantages of using TestingBot is its `concurrency`: run multiple tests simultaneously to drastically shorten the total time required to run all your tests. For example: - Running 10 tests in the build `sequentially`, where each test takes 30 seconds, would result in a total test build duration of **300 seconds**. - Running these 10 tests in `parallel`, where each test still takes 30 seconds, would result in a total test build duration of **30 seconds**. The `efficiency percentage` is calculated as a percentage. It is based on how long the build took to run, compared to the duration of the longest test in the build. Efficiency Percentage | Degree of Parallelization | Description || 0%-20% | Sequential | All your tests have been running sequentially. Consider speeding up your build time with parallelization. | | 20%-90% | Semi-Parallel | Some of your tests are running in parallel. There is still room for improvement: add more parallelization to run all tests in your build simultaneously. | | 90%-100% | Full Parallel | You are taking full advantage of parallelization and your test build is optimised for the fastest build time. | ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/analytics/guide # Analytics TestingBot provides detailed analytics about your test sessions. We provide you with graphs and metrics, allowing you to: - Reduce time spent analyzing test results - Locate bottlenecks in your test performance - See for which browsers and platforms your tests are flaky and more prone to failure - Determine if a build is release quality ## Filters Our analytics filters allow you to drill down on your metrics and visualize the information you need. ![Analytics Filter](https://testingbot.com/assets/support/analytics/filter-90b35e678a8aa60b19d46a1f1edd49d4f6c01c8b069da8b4a2b5e570a32a8e9e.png) You can filter on several things, including: - Start/End time of the test - Browser/Device - Status (Passed, Failed, Unknown) - Group by date/status - Filter by [Build](https://testingbot.com/support/web-automate/selenium/test-options#build) - Filter by groups: only show tests that have a [group (tag)](https://testingbot.com/support/web-automate/selenium/test-options#grouping) included in their desired capability. - Filter by [team member](https://testingbot.com/support/team/sub-accounts) ## Graphs ![Analytics Graphs](https://testingbot.com/assets/support/analytics/graph-07d678a57d51edf51c1fafa266ddfef3bb4264ba6dccf334d5b372b0a35954b4.png) One of the first graphs you'll see is the graph above. This Column Chart shows an overview of the pass/fail rate for all your tests, grouped by day, for the last 31 days. Each column represents a day, and may include: - **Passed/Success:** all tests marked as passed - **Failure:** all tests marked as failed - **Unknown:** we don't know the status about this test. Please use our API to mark this test as passed/failed. Using the filter, you can change the date range and filter by various things (see above). If you click part of the graph, you'll navigate to an overview of all tests for that specific part. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/billing # Billing Everything about paying for TestingBot: how billing works, the payment methods we accept, and how to manage or cancel your subscription. - [Payment Options](https://testingbot.com/support/billing/payment-options): credit card, PayPal, wire transfer, and AWS Marketplace. - [Charges](https://testingbot.com/support/billing/charges): how the monthly and annual billing cycle works. - [Upgrading](https://testingbot.com/support/billing/upgrading): change your plan or top up with Pay As You Go. - [Invoices](https://testingbot.com/support/billing/invoices): download invoices and set your company details and VAT number. - [Cancel](https://testingbot.com/support/billing/cancel): stop your subscription at any time. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/billing/payment-options # Payment Options You can see the list of our prices on our [pricing page](https://testingbot.com/pricing). We accept the following payment options: - **Credit card:** handled securely by Stripe - **PayPal:** available through Stripe - **Wire transfer and purchase orders:** for annual and enterprise plans - **EU local methods:** SEPA Direct Debit and iDEAL, through Stripe - **AWS Marketplace:** buy through your existing AWS account TestingBot is PCI compliant. Please view our [SAQ A](https://testingbot.com/pci.pdf). ## Credit Card ![supported credit cards](https://testingbot.com/assets/cards-1c130150c727d3ba2045f3249ed5e069c1f42ccc17cb858689a41a1ccb7411bf.png) Billing occurs as a subscription and is handled by Stripe.com. PayPal and EU local methods such as SEPA Direct Debit and iDEAL are also processed through Stripe. ![Stripe Logo](https://testingbot.com/assets/stripe-7c2d0c43d09cb95597fb1402e2e35ac340f158502298ed5dfb4234ecf2c4e993.png) All payments happen on a HTTPS secured page. **Your payment data is never stored or logged on our website.** ## AWS Marketplace TestingBot is available on the AWS Marketplace. This is a convenient option for teams that already buy through AWS and want testing on the same bill. - Pay through your existing AWS account, with no new vendor to set up - One consolidated AWS invoice for your finance team - Faster procurement, since AWS is already an approved vendor For custom or multi-year terms, [contact us](https://testingbot.com/contact/new) to arrange an offer through the Marketplace. [View TestingBot on AWS Marketplace](https://aws.amazon.com/marketplace/pp/prodview-yv6ds4mf77fao) ## Wire Transfer and Purchase Orders For annual and enterprise plans we also accept wire transfer and purchase orders. [Contact us](https://testingbot.com/contact/new) for bank details and to arrange payment. ## Invoices and VAT Invoices for your subscription are available in your [account billing](https://testingbot.com/members/invoices) section. See the [Invoices](https://testingbot.com/support/billing/invoices) page for how to add your company details and VAT number. Customers in the EU are charged VAT at checkout. Businesses elsewhere in the EU with a valid VAT ID are reverse charged and pay no VAT. Belgian businesses are charged Belgian VAT. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/billing/charges # Charges Billing occurs as a subscription, monthly or annual. Each cycle your account is recharged with testing minutes. How many minutes are added depends on the subscription you choose. You can see the various subscriptions on our [pricing page](https://testingbot.com/pricing). ## What if I use more than the provided test minutes? You will no longer be able to run your tests until your minutes refresh at the start of your next billing period. **We will never automatically re-charge your subscription for extra minutes, no hidden costs.** ## Need more minutes? Pay As You Go If you need extra minutes before your next billing date, you can buy a one-time [Pay As You Go](https://testingbot.com/pricing) credit pack. The minutes are added to your account right away, with no change to your subscription. To raise your allowance every cycle, [upgrade your plan](https://testingbot.com/support/billing/upgrading) instead. ## When will I be billed? Monthly subscriptions are billed on the same day each month. If you started on the 11th, you are charged on the 11th of the following month. Annual subscriptions are billed once a year on your renewal date. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/billing/upgrading # Upgrading subscription At any time you can [upgrade your current subscription](https://testingbot.com/pricing). You will not lose your remaining testing minutes for that month. Minutes will be transferred over immediately when you upgrade your plan. The number of simultaneous VMs allowed will be automatically increased after your upgrade. ## Downgrading To move to a smaller plan, select it on the [pricing page](https://testingbot.com/pricing). If you need help changing or reducing your plan, [contact us](https://testingbot.com/contact/new). To stop your subscription entirely, see [Cancel Subscription](https://testingbot.com/support/billing/cancel). ## Monthly or annual billing Plans are available on a monthly or annual basis. Annual billing is discounted, and you can choose it on the [pricing page](https://testingbot.com/pricing). ## Pay As You Go If you only need extra minutes occasionally, you can buy a one-time [Pay As You Go](https://testingbot.com/pricing) credit pack instead of upgrading. The minutes are added to your account right away. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/billing/invoices # Invoices We provide invoices for all customers with an active subscription. You can find monthly/yearly invoices (depending on the subscription) in the member area under [Billing](https://testingbot.com/members/invoices). Each invoice can be downloaded as a PDF from the Invoices table. If you are a sub-account in a team account, you will not be able to see the invoices. Only parent accounts or standalone accounts can see invoices. ## Changing Billing Details To change your billing details (Company name, address, VAT number, ...) please go to [Account Settings](https://testingbot.com/members/user/edit) and update the fields on the Profile tab. When you update your billing details, future invoices will reflect the new information. Re-download the invoice from the Billing page to get the updated copy. ## VAT number If your company is located in the EU, add your VAT number on the Profile tab of your [Account Settings](https://testingbot.com/members/user/edit), or enter it at checkout. Customers in the EU are charged VAT at checkout. Businesses elsewhere in the EU with a valid VAT number are reverse charged and pay no VAT, which is reflected on the invoice. Belgian businesses are charged Belgian VAT. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/billing/cancel # Cancel Subscription You can stop your subscription at any time. Go to [Billing](https://testingbot.com/members/invoices) in the member area and click the **Cancel Subscription** button under "Manage your Subscription". Your account remains active until the end of your current billing period. You keep access to all features until then. Customers who subscribed through [AWS Marketplace](https://aws.amazon.com/marketplace/pp/prodview-yv6ds4mf77fao) should cancel through their AWS Marketplace subscription page. Sorry to see you go. Please [let us know](https://testingbot.com/contact/new) how we can improve our service. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/parallel-calculator # Parallel Testing Calculator Parallel testing allows you to run multiple tests simultaneously across various browsers, devices, and OS combinations. This approach enables more tests to be run at once, significantly decreasing overall testing time. Faster build times mean faster releases and less time spent waiting for builds to complete. Use this calculator to understand how parallel testing can help you achieve your test coverage and build execution time goals. Simply input your current testing setup and desired build time to calculate the optimal number of parallel tests needed. | Team | Current State | Desired Future State | Recommendation | | --- | --- | --- | --- | | # of Browsers and/or Devices (Current) | # of Parallel Tests | Build Time (hours) | # of Browsers and/or Devices (Desired) | Build Speed | | --- | --- | --- | --- | --- | | Team 1 | | | | | Select... Supersonic (15m) Fast (30m) Basic (2h) Custom | - | | + Add Team Maximum of 10 teams reached. | Total Recommendation: - | Calculate Save & Share Use this URL to view & share these results: Copy ## Understanding the Calculator - **# of Browsers and/or Devices:** The number of unique desktop browser and/or mobile device combinations to test. - **# of Parallel Tests:** The number of parallel tests you are currently running. - **Build Time:** Time taken for your entire build to complete. --- URL: https://testingbot.com/support/team # TestingBot Team Management TestingBot provides team accounts, allowing you to share test results with your colleagues. - [Account Settings](https://testingbot.com/support/team/account) - [Team & Sub-Accounts](https://testingbot.com/support/team/sub-accounts) - [Roles & Permissions](https://testingbot.com/support/team/rbac) - [Roles & Permissions Reference](https://testingbot.com/support/team/rbac/roles-and-permissions) - [Custom Roles](https://testingbot.com/support/team/rbac/custom-roles) - [Member Roles](https://testingbot.com/support/team/rbac/member-roles) - [Product Access](https://testingbot.com/support/team/rbac/product-access) - [Service Accounts](https://testingbot.com/support/team/service-accounts) - [Audit Logs](https://testingbot.com/support/team/audit-logs) - [2FA](https://testingbot.com/support/team/2fa) - [Enforce 2FA](https://testingbot.com/support/team/enforce-2fa) - [SSO](https://testingbot.com/support/team/sso) - [Microsoft Entra](https://testingbot.com/support/team/sso/microsoft-entra) - [Okta](https://testingbot.com/support/team/sso/okta) --- URL: https://testingbot.com/support/team/sub-accounts # Managing Sub-Accounts Sub-accounts are accounts that can be linked to your account. They're a great way to share your subscription and test results with your team members. You can invite several people at once, assign each invitation a role, and manage invitations that are still pending. - Everyone in the same group can see each other's tests. - Each sub-account has its own login credentials to log into our TestingBot member dashboard, and they have their own key/secret credentials to run tests on our Selenium grid. A sub-account can create/run/delete tests, see test results and interact with the member dashboard. The things **sub-accounts can't do** : - Access any subscription related items: payment information, invoices, accounting details, ... - Add/remove any other sub-accounts (unless they have the Admin role) ## Adding Sub-Accounts Go to [My Account \> Add a team member](https://testingbot.com/members/sub/new) to invite people to your team. Enter one or more email addresses in the invite field (press Enter, comma, or space after each one to confirm it), choose a [role](https://testingbot.com#permissions), then click **Send invitations**. You can invite up to 10 people at a time this way. The same role is applied to everyone in the submission. What happens next depends on the email address: - **The person already has a TestingBot account:** they receive an invitation to join your team. It stays under [Pending Invitations](https://testingbot.com#pending) until they accept. - **The email address is new to TestingBot:** we create a sub-account for them right away and email a link so they can set a password and log in. ### Inviting a large team To add more people at once, use **Or upload a CSV** and select a `.csv` file with one email address per line. The role you selected is applied to every address in the file. ## Roles and Permissions Every team member now has **two roles** that work together: - **IAM role** (Owner, Admin, or User): administrative authority. It decides who can manage the account and grant access, for example managing team members, billing, security, and integrations. The IAM role is derived from the member's organization role and is fixed. - **RBAC role** (Admin, User, Viewer, or a custom role): product capability. It decides what the member can do in the products, such as running, viewing, or deleting tests. Custom roles are RBAC only and can never grant administrative authority. For the full model, permission matrix, and how product access resolves, see [Role-based access control](https://testingbot.com/support/team/rbac) and the [Roles and permissions reference](https://testingbot.com/support/team/rbac/roles-and-permissions), which are the source of truth. You can change a member's roles later by editing them on the [Team Members](https://testingbot.com/members/sub) page. ## Pending Invitations Invitations sent to people who already have a TestingBot account appear under **Pending Invitations** on the [Team Members](https://testingbot.com/members/sub) page until they accept. For each invitation you can see the email address, who sent it, and the role it grants. Admins can manage a pending invitation: - **Resend:** send the invitation email again. - **Revoke:** cancel the invitation. Sub-accounts created for brand-new email addresses do not appear here, because they become team members straight away. ## Modifying Sub-Accounts To edit or remove a sub-account, go to [My Account \> Team Members](https://testingbot.com/members/sub) and use the actions menu next to a member. From the edit screen you can change their role, read-only access, and concurrency limit. **Modify access** now sets both an IAM role and an RBAC role for the member. See [Assign member roles](https://testingbot.com/support/team/rbac/member-roles) for details on choosing each one. ### Disconnecting a sub-account You can also **disconnect a sub-account** from the actions menu. This disconnects the sub-account from your group. The account will not be removed from our system, but it will no longer be associated with your account or other sub-accounts. ## Read-Only (Viewer) Accounts A **Viewer** is a read-only sub-account: it can view tests and results but cannot modify or remove any results from the team account. You can assign the Viewer role when inviting someone, or switch an existing member to read-only by editing them on the Team Members page. ## Exporting Members Admins can export the team to a CSV file. On the [Team Members](https://testingbot.com/members/sub) page, click **Export Members** and choose what to include: - **All:** current members and pending invitations. - **Members:** current team members only. - **Invitations:** pending invitations only. The file downloads as `team_members_.csv`. ## Concurrency Limit The Concurrency Limit indicates how many tests a user can run simultaneously. The more tests you run concurrently, the faster you can process all your tests. By default, a sub-account will have the same amount of concurrency as the parent account (which depends on the pricing plan that the parent account is subscribed to). It is possible to change the concurrency limit per user. This way, you can for example divide the amount of concurrency you have across various team members. For example: you have an Enterprise plan with 30 parallel tests: You can assign 10 parallel tests to team member **A** , 15 to team member **B** and 5 to team member **C**. A user can not run more tests concurrently than the maximum amount assigned. If the user tries to run more tests, the tests will be queued in our system, waiting for a slot to free up. If you do not change this setting for any of the sub-accounts, then all sub-accounts will have access to the maximum amount of concurrency. Whenever this threshold is reached, extra tests for any sub-account will be queued until a slot frees up. ## Tests Overview for Sub-Accounts To see a list of tests created by a sub-account, click "Tests run by this user". The list contains tests run by this sub-account (with their own key/secret). ## Team Activity Log Team admins are able to see an overview of the actions performed by their team members in the Team Activity Log overview page. On the Team Management page, please click the **Activity Log** tab to open the overview. In this activity log we will show the following events for your team members: - A user from your team logged in - A user from your team logged out - A user was invited to your team - An invitation was accepted or declined - A user was added to your team - A user was removed from your team - A team member's user level was upgraded or downgraded ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/team/2fa # Why Two-Factor Authentication (2FA) Two Factor Authentication adds an extra layer of security to your TestingBot account. After entering your email and password, you'll also need to supply a one-time verification code. TestingBot supports two methods for generating that code: a time-based authenticator app (TOTP) or an SMS message to your mobile phone. We recommend an authenticator app — it works without cell coverage and is generally more secure than SMS. With 2FA enabled, even if your password is compromised, an attacker would still need access to your authenticator app or phone to sign in. Team admins can require 2FA for everyone on the team. See [Enforce Two-Factor Authentication](https://testingbot.com/support/team/enforce-2fa). ## Enable 2FA Log into your TestingBot account and go to [Security](https://testingbot.com/members/user/security) in your account settings. Click the [Enable 2FA](https://testingbot.com/members/2fa/enable) button to begin the setup. On the setup page, you can choose either method: - **Authenticator app (recommended):** scan the QR code with any TOTP-compatible app such as Google Authenticator, Authy, 1Password or Microsoft Authenticator, then enter the 6-digit code it generates to verify. - **SMS:** enter your mobile phone number with country code (for example, a US number starts with `+1`). We'll send a verification code via SMS that you enter to verify. Once 2FA is enabled, every subsequent login will prompt you for a verification code after your email and password. ## Authenticator App Any TOTP-compatible authenticator app will work with TestingBot 2FA, including Google Authenticator, Authy, 1Password, and Microsoft Authenticator. Scan the QR code shown on the [Enable 2FA](https://testingbot.com/members/2fa/enable) page to register TestingBot with your app. The app will then generate a new 6-digit code every 30 seconds, which you enter when signing in. ## Disabling 2FA Log into your TestingBot account and go to [Security](https://testingbot.com/members/user/security) in your account settings. Click the **Disable 2FA** button to turn off two-factor authentication for your account. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/team/enforce-2fa # Enforce Two-Factor Authentication As a team admin you can require every member of your team to use [two-factor authentication (2FA)](https://testingbot.com/support/team/2fa) when signing in to TestingBot. When enforcement is on, members who have not set up 2FA are guided through setup before they can use the dashboard, so your whole team is protected by a second factor, not just the people who opted in. Enforcement is configured from the **Authentication** tab on your team management page. Any team admin (the account owner or a member with the Admin role) can turn it on or off. Two-factor authentication is a feature of our paid plans. ## Turn on enforcement Open the [Authentication](https://testingbot.com/members/authentication) tab under your team management page (Members dashboard, then **Team → Authentication** ), and follow these steps: ![Enforce two-factor authentication card with the Configure button](https://testingbot.com/assets/support/team/enforce-2fa/enforce-card-5a50c180f687820576ddbecbbb287ba8721cc27930a4aa70be74aa9bc1084feb.webp) - Click **Configure** on the "Enforce two-factor authentication" card. - Review the effects in the confirmation dialog. - Click **Confirm**. ![Enforce two-factor authentication confirmation dialog](https://testingbot.com/assets/support/team/enforce-2fa/confirm-modal-6af63e3163df42a8e45b671c7229949590d0e97c0839900176b2a4490371ca17.webp) Every member who does not yet have 2FA is automatically emailed with instructions to set it up. The card then shows that enforcement is active, along with how many members have completed setup. ![Authentication tab showing two-factor authentication is enforced](https://testingbot.com/assets/support/team/enforce-2fa/enforced-card-3372fea70bcbb347e9d76971753654459a6c65306952484c56928e67e7c961a2.webp) ## What happens when enforcement is on - **Everyone must comply.** All human team members, including admins and the account owner, are required to use 2FA. [Service accounts](https://testingbot.com/support/team/service-accounts) are API-only and cannot sign in to the dashboard, so they are excluded. - **Current members get 7 days.** Members who already belong to the team have a 7-day grace period from the moment enforcement is enabled. After that, they are asked to set up 2FA before they can continue using the dashboard, and are prompted to set it up at their next sign in. - **New members set up immediately.** Anyone added to the team after enforcement is enabled must set up 2FA before they can use their account. - **Pending members are flagged.** On the [Team Members](https://testingbot.com/members/sub) list, anyone who has not yet set up 2FA shows a `2FA pending` badge, so you can see compliance at a glance. ![Team Members list showing 2FA pending badges](https://testingbot.com/assets/support/team/enforce-2fa/members-pending-ef05a1a164f5a9dbd00ba2acffa8d3c5b1a31d7b1840fc60afbc826154afd6aa.webp) Members set up 2FA through the normal flow described in [Two-Factor Authentication](https://testingbot.com/support/team/2fa): an authenticator app (recommended) or SMS. ## Reset a member's 2FA If a member loses access to their authenticator app or phone, a team admin can reset their 2FA. On the [Team Members](https://testingbot.com/members/sub) list, open the actions menu (`…`) next to the member and choose **Reset 2FA**. ![Reset 2FA option in the team member actions menu](https://testingbot.com/assets/support/team/enforce-2fa/reset-menu-42f45415aa9c7da668c7dff8f1001a86331a314fedbd1809ca1cc32bd65a7ed2.webp) The member's existing 2FA is removed and they are emailed about the change. The next time they sign in they are prompted to set up two-factor authentication again. Their old authenticator entry for TestingBot will no longer work, so they should remove it and add a fresh one during setup. ![Reset 2FA confirmation dialog](https://testingbot.com/assets/support/team/enforce-2fa/reset-modal-a2fd46081e728f0cd5f5278b2a94977b1024604964ee61b9e690af1359f58b06.webp) ## Disable enforcement To stop requiring 2FA across your team, open the [Authentication](https://testingbot.com/members/authentication) tab and click **Disable enforcement**. Members who already enabled 2FA keep it; they are simply no longer forced to use it. You can re-enable enforcement at any time. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/team/rbac # Role-based access control Role-based access control (RBAC) lets you control what each member of your team can see and do. TestingBot uses two types of roles. One type governs administrative authority over the account. The other governs which product capabilities a member can use. Keeping these separate lets you grant someone broad product access without making them an account administrator, or the other way around. Role-based access control is available on the Enterprise plan. This includes the IAM and RBAC roles, custom roles, and per-member access management described on these pages. On other plans, the [Roles & Permissions](https://testingbot.com/members/roles) page shows the built-in roles with an option to upgrade. [See plans](https://testingbot.com/pricing). This page is an overview. It explains the two role types, the built-in roles, where to manage them, and how TestingBot decides whether a given action is allowed. ## The two role types Every team member has exactly one role of each type. The two types answer different questions and do not overlap. - **IAM (Identity & Access Management): administrative authority.** This type decides who can manage the account itself, such as inviting and removing members, changing roles, managing billing and the subscription, and configuring team-wide settings. The IAM roles are **Owner** , **Admin** , and **User**. The Owner has full control of the account. Admins can manage most team and account settings. Users have no administrative authority. - **RBAC (Role-Based Access Control): product capability.** This type decides what a member can do with the testing products, such as running and managing tests, viewing results, and changing test data. The RBAC roles are **Admin** , **User** , and **Viewer** , plus any **custom roles** you create. An Admin has full product capability. A User can run and manage tests but cannot delete them. A Viewer has read-only access. Because the two role types are independent, a person can be an IAM **User** with no administrative power while holding an RBAC **Admin** role with full product capability, or any other combination that fits their responsibilities. ## Built-in roles Each role type ships with a fixed set of built-in roles. Built-in roles cannot be edited or removed. If you need finer-grained product permissions, create a [custom role](https://testingbot.com/support/team/rbac/custom-roles) for the RBAC product capability. | Type | Role | Description | | --- | --- | --- | | IAM | **Owner** | Full control of the account, including billing, the subscription, and all members. There is one Owner per account. | | IAM | **Admin** | Manages team members, roles, and most account settings. Cannot transfer ownership of the account. | | IAM | **User** | No administrative authority. Belongs to the team but cannot manage members, roles, or billing. | | RBAC | **Admin** | Full product capability. Can run, manage, and delete tests and test data across the team. | | RBAC | **User** | Can run and manage tests. Cannot delete tests or test data. | | RBAC | **Viewer** | Read-only access. Can view tests and results but cannot run or modify anything. | ## Where to manage roles Admins open Team Management and select the **Roles & Permissions** tab at [/members/roles](https://testingbot.com/members/roles) to review the built-in roles and to create custom RBAC roles. ![Roles and Permissions page showing IAM roles and RBAC roles](https://testingbot.com/assets/support/team/rbac/roles-index-7628470f310d9a99d57bc267285f53b8baad753b33fb418725e71cbd13f0a10d.png) Each team member shows both an IAM role and an RBAC role on the [Team Members](https://testingbot.com/members/sub) page, where you can change either role for an individual member. Custom RBAC roles let you define your own permission sets. See [Custom roles](https://testingbot.com/support/team/rbac/custom-roles) for details. ## How access is decided When a member attempts an action, TestingBot resolves both roles. - The **IAM role** is resolved from the member's role in the account (Owner, Admin, or User). This determines whether an administrative action is allowed. - The **RBAC role** is resolved from the member's tier (Admin, User, or Viewer) or from the custom role assigned to them. This determines which product capabilities are granted. - Product access also depends on your **plan**. A capability is available only when the RBAC role grants it and the plan includes it. Both conditions must be true, so a permission the plan does not cover is never available regardless of role. The **Owner** always has full access and is never restricted by RBAC roles. ## Learn more - [Roles & permissions reference](https://testingbot.com/support/team/rbac/roles-and-permissions): the full permission reference for the built-in roles. - [Custom roles](https://testingbot.com/support/team/rbac/custom-roles): define your own RBAC permission sets. - [Member roles](https://testingbot.com/support/team/rbac/member-roles): assign IAM and RBAC roles to individual team members. - [Product access](https://testingbot.com/support/team/rbac/product-access): how plans and RBAC roles combine to grant capabilities. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/team/rbac/roles-and-permissions # Roles and permissions reference This page is the authoritative catalog of every permission in TestingBot. Permissions are split across two role types. RBAC permissions control what you can do inside the testing products. IAM permissions control who can administer the account and grant access to others. Each permission has a stable key (for example `tests.run`). The tables below show the exact key for every permission and which built-in role grants it. For an introduction to how these two role types fit together, see [Role-based access control](https://testingbot.com/support/team/rbac). To manage assignments, open [Roles & Permissions](https://testingbot.com/members/roles) and [Team Members](https://testingbot.com/members/sub). Role-based access control is available on the Enterprise plan. [See plans](https://testingbot.com/pricing). ## RBAC permissions RBAC permissions describe product and entity capabilities: what a member can do in Live Testing, Automated Testing, Visual Testing, and the other products. They are resolved from the RBAC tier (Admin, User, or Viewer) or from an assigned custom role. The account Owner and service accounts always receive full RBAC capability. A "✓" means the role's bundle grants the permission. An empty cell means it does not. | Permission | Key | Admin | User | Viewer | | --- | --- | --- | --- | --- | | Products | | --- | | Live Web Testing | `product.live_web.access` | ✓ | ✓ | ✓ | | Live App Testing | `product.live_app.access` | ✓ | ✓ | ✓ | | Automated Testing | `product.automation.access` | ✓ | ✓ | ✓ | | Visual Testing | `product.visual.access` | ✓ | ✓ | ✓ | | Accessibility Testing | `product.accessibility.access` | ✓ | ✓ | ✓ | | AI Testing | `product.ai.access` | ✓ | ✓ | ✓ | | Tests | | --- | | View tests | `tests.view` | ✓ | ✓ | ✓ | | Run tests | `tests.run` | ✓ | ✓ | | | Delete tests | `tests.delete` | ✓ | | | | Integrations | | --- | | View integrations | `integrations.view` | ✓ | ✓ | ✓ | | Reports | | --- | | View reports | `reports.view` | ✓ | ✓ | ✓ | The six `product.*` permissions are necessary but not sufficient: a member can only use a product when both the role grants the product permission and the plan includes that product. In the role editor, products that the plan does not include show a "plan" badge. A member's effective product access is therefore the role permission combined with plan inclusion. For more detail, see [Product access](https://testingbot.com/support/team/rbac/product-access). ## IAM permissions IAM permissions describe administrative authority: who can manage the account, manage team members, manage billing, and grant access to others. They are derived from the organization role (Owner, Admin, or User) and are fixed, so they cannot be customized. A "✓" means the role grants the permission. An empty cell means it does not. | Permission | Key | Owner | Admin | User | | --- | --- | --- | --- | --- | | Team | | --- | | View team members | `team.members.view` | ✓ | ✓ | | | Manage team members | `team.members.manage` | ✓ | ✓ | | | Manage roles | `team.roles.manage` | ✓ | ✓ | | | Billing | | --- | | View billing | `billing.view` | ✓ | ✓ | | | Manage billing | `billing.manage` | ✓ | ✓ | | | Account | | --- | | View account settings | `account.settings.view` | ✓ | ✓ | | | Manage account settings | `account.settings.manage` | ✓ | ✓ | | | Transfer ownership | `account.transfer_ownership` | ✓ | | | | Delete account | `account.delete` | ✓ | | | | Manage integrations | `integrations.manage` | ✓ | ✓ | | | Manage service accounts | `service_accounts.manage` | ✓ | ✓ | | | Manage security | `security.manage` | ✓ | ✓ | | Note that `integrations.manage` is an IAM permission (who can configure integrations for the account), while `integrations.view` in the RBAC table above is a separate product capability. For background on these two role types, see [Member roles](https://testingbot.com/support/team/rbac/member-roles) and [Service accounts](https://testingbot.com/support/team/service-accounts). ## Key differences - A **Viewer** (RBAC) can view, but cannot run or delete tests: both `tests.run` and `tests.delete` are withheld. This is the read-only product role. - A **User** (RBAC) can do everything a member needs day to day, but cannot delete tests: `tests.delete` is reserved for Admin. - An **Admin** (IAM) can manage the team, billing, account settings, integrations, service accounts, and security, but cannot transfer ownership or delete the account: `account.transfer_ownership` and `account.delete` are reserved for the Owner. - A **User** (IAM) has no administrative authority at all. IAM permissions are fixed and derived from the organization role, so they cannot be customized. - **Custom roles are RBAC-only.** They can grant or withhold any RBAC permission, but can never grant IAM permissions. See [Custom roles](https://testingbot.com/support/team/rbac/custom-roles). Related reading: [Roles and permissions overview](https://testingbot.com/support/team/rbac/roles-and-permissions), [Sub-accounts](https://testingbot.com/support/team/sub-accounts), [Two-factor authentication](https://testingbot.com/support/team/2fa), [Enforce two-factor authentication](https://testingbot.com/support/team/enforce-2fa), and [Audit logs](https://testingbot.com/support/team/audit-logs). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/team/rbac/custom-roles # Manage custom roles Custom roles let you define granular role-based access control (RBAC) roles for your team. Instead of using only the built-in Admin, User, and Viewer tiers, you can build a role that grants exactly the products, test actions, integration views, and reports each member needs. Role-based access control, including custom roles, is available on the Enterprise plan. On other plans, the [Roles & Permissions](https://testingbot.com/members/roles) page shows the built-in roles with an option to upgrade. [See plans](https://testingbot.com/pricing). Custom roles are RBAC-only: they control what a member can do inside the products. They can never grant administrative (IAM) permissions such as managing billing, integrations, or other team members. For the difference between the two role types, see [Role-Based Access Control](https://testingbot.com/support/team/rbac) and [Roles & Permissions](https://testingbot.com/support/team/rbac/roles-and-permissions). ## Create a custom role Go to [Roles & Permissions](https://testingbot.com/members/roles) and click **New role**. The role editor has the following fields: - **Role name:** a short, descriptive name shown when you assign the role to a member. - **Base tier:** the starting tier for the role, one of Admin, Member, or Viewer. This sets the ceiling for the non-product permissions the role can include. - **Description:** an optional note explaining what the role is for. - **Permissions:** a grid of permissions to enable, grouped by Products, Tests, Integrations, and Reports. ![New custom role form with base tier and permission checkboxes](https://testingbot.com/assets/support/team/rbac/custom-role-form-36ed17830b116911fd81abc0d80ca21fda912b58c5b61399ffad0dcd87468eea.png) The permissions grid covers the full RBAC set: - **Products:** Live Web Testing, Live App Testing, Automated Testing, Visual Testing, Accessibility Testing, and AI Testing. Each product also requires your plan to include that product, shown by a **plan** badge in the editor. - **Tests:** View tests, Run tests, and Delete tests. - **Integrations:** View integrations. - **Reports:** View reports. For reference, the built-in tiers bundle these permissions as follows: | Permission | Admin | User | Viewer | | --- | --- | --- | --- | | Product access (all 6 products) | ✓ | ✓ | ✓ | | View tests | ✓ | ✓ | ✓ | | Run tests | ✓ | ✓ | · | | Delete tests | ✓ | · | · | | View integrations | ✓ | ✓ | ✓ | | View reports | ✓ | ✓ | ✓ | Admin includes all 11 RBAC permissions. User includes everything except Delete tests. Viewer is read-only, with everything except Run tests and Delete tests. ## Base tier and ceiling The base tier you select caps which non-product permissions the role can include. A role with a Viewer base tier cannot exceed Viewer, so it cannot grant Run tests or Delete tests no matter what you select in the grid. Choose a base tier that matches the highest level of access the role should reach. Product permissions are not capped by the base tier. Instead, each product permission is gated by your plan: the member gets access to a product only when your plan includes that product **and** the role grants `product..access`. A member's effective product access is therefore the plan and the role combined. ## Rules and limits To keep custom roles safe, the editor enforces a few rules: - You can only grant permissions you hold yourself. Permissions you do not have are greyed out in the grid. - Custom roles cannot grant administrative (IAM) permissions. They are RBAC-only and never include things like managing integrations, billing, or team members. - Owner-only permissions cannot be granted by a non-owner admin. - Only the account owner can create Admin-tier roles. The account owner and service accounts always have full RBAC capability and are not limited by these roles. ## Assign a custom role Once a custom role exists, assign it to a member using **Modify access** on the [Team Members](https://testingbot.com/members/sub) page. Your custom roles appear in the same list as the built-in tiers. For the full assignment workflow, see [Member roles](https://testingbot.com/support/team/rbac/member-roles). ## Edit or delete You can edit or delete a custom role from the [Roles & Permissions](https://testingbot.com/members/roles) page. - **Editing** a role updates every member who holds it immediately. Adding or removing a permission changes what those members can do right away. - **Deleting** a role reverts its members to a built-in role, so no one is left without any access. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/team/rbac/member-roles # Assign member roles Every person on your TestingBot team has two roles that work together. An **IAM role** controls administrative authority: who can manage the account, billing, and team. An **RBAC role** controls product capability: what they can do inside the testing products. Both roles are shown for each person on the [Team Members](https://testingbot.com/members/sub) page. This guide explains how to change them. For the full breakdown of every permission, see [Roles & Permissions](https://testingbot.com/support/team/rbac/roles-and-permissions) and the [RBAC overview](https://testingbot.com/support/team/rbac). Role-based access control is available on the Enterprise plan. [See plans](https://testingbot.com/pricing). ## Modify access Go to the [Team Members](https://testingbot.com/members/sub) page, open the actions menu next to the member you want to update, and choose **Modify access**. A dialog opens with separate selectors for the IAM role and the RBAC role, plus the member's concurrency limit. ![Modify access dialog with IAM role and RBAC role selectors](https://testingbot.com/assets/support/team/rbac/modify-access-48348558dcd157e304b4fe5249dce9273599555b3c659414f3961fff3f0d8aef.png) Adjust the role selectors as needed and save. The new permissions apply immediately on the member's next request. ## IAM role The IAM role decides how much administrative authority a member has over the account itself. Choose **Owner** , **Admin** , or **User**. Only the account owner can change a member's IAM role, and IAM roles are fixed (they cannot be customized). - **Owner:** full administrative authority over the account, including transferring ownership and deleting the account. There is one owner per account. - **Admin:** can manage team members, roles, billing, account settings, integrations, service accounts, and security. An Admin cannot transfer ownership or delete the account. - **User:** no administrative authority. A User can still work in the products according to their RBAC role, but cannot manage the team, billing, or account settings. The table below shows which administrative permissions each IAM role grants. | Administrative permission | Owner | Admin | User | | --- | --- | --- | --- | | View team members | ✓ | ✓ | · | | Manage team members | ✓ | ✓ | · | | Manage roles | ✓ | ✓ | · | | View billing | ✓ | ✓ | · | | Manage billing | ✓ | ✓ | · | | View account settings | ✓ | ✓ | · | | Manage account settings | ✓ | ✓ | · | | Manage integrations | ✓ | ✓ | · | | Manage service accounts | ✓ | ✓ | · | | Manage security | ✓ | ✓ | · | | Transfer ownership | ✓ | · | · | | Delete account | ✓ | · | · | ## RBAC role The RBAC role decides what a member can do inside the products. Choose **Admin** , **User** , or **Viewer**. On Enterprise plans you can also assign a [custom role](https://testingbot.com/support/team/rbac/custom-roles). Any team admin can change a member's RBAC role. - **Admin:** full product capability, including running and deleting tests across every product your plan includes. - **User:** can view and run tests, but cannot delete tests. - **Viewer:** read-only. A Viewer can view tests, reports, and integrations, but cannot run or delete tests. The table below shows which product capabilities each RBAC role grants. | Product capability | Admin | User | Viewer | | --- | --- | --- | --- | | Live Web Testing | ✓ | ✓ | ✓ | | Live App Testing | ✓ | ✓ | ✓ | | Automated Testing | ✓ | ✓ | ✓ | | Visual Testing | ✓ | ✓ | ✓ | | Accessibility Testing | ✓ | ✓ | ✓ | | AI Testing | ✓ | ✓ | ✓ | | View tests | ✓ | ✓ | ✓ | | Run tests | ✓ | ✓ | · | | Delete tests | ✓ | · | · | | View integrations | ✓ | ✓ | ✓ | | View reports | ✓ | ✓ | ✓ | Each product capability also requires your plan to include that product. A member's effective product access is the combination of the plan including the product **and** the RBAC role granting access to it. The account owner and [service accounts](https://testingbot.com/support/team/service-accounts) always have full RBAC capability. Custom roles are RBAC-only and can never grant IAM permissions. ## Concurrency limit From the same **Modify access** dialog you can set how many parallel tests the member is allowed to run. This caps the member's share of your account's total parallel capacity. For details on how concurrency is allocated across the team, see the concurrency section in [Team & Sub-Accounts](https://testingbot.com/support/team/sub-accounts#concurrency). ## Auditing Every role change is recorded so you always know who changed what and when. Review these entries in the [Activity Log](https://testingbot.com/support/team/audit-logs). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/team/rbac/product-access # Manage product access Product access controls which TestingBot products a role can use: Live Web Testing, Live App Testing, Automated Testing, Visual Testing, Accessibility Testing, and AI Testing. You manage these permissions per role under [Roles & Permissions](https://testingbot.com/members/roles), then assign roles to your [Team Members](https://testingbot.com/members/sub). Product access is part of TestingBot's role-based access control (RBAC). For the full picture, see [Role-Based Access Control](https://testingbot.com/support/team/rbac) and [Roles & Permissions](https://testingbot.com/support/team/rbac/roles-and-permissions). Role-based access control is available on the Enterprise plan. [See plans](https://testingbot.com/pricing). ## Two gates Whether a team member can open a product depends on two gates. A member gets access only when **both** are satisfied: - **Your plan includes the product.** Each product is part of your TestingBot subscription. If your plan does not include a product, no role can grant access to it. - **The role has the matching permission.** The role assigned to the member must include the relevant `product..access` permission. In the role editor, each product permission shows a **plan** badge to indicate the plan gate. The badge is a reminder that the permission only takes effect when your plan provides that product. ![Products section of the role editor with plan badges](https://testingbot.com/assets/support/team/rbac/product-access-9097ef6d47c0b822914c3a5657dd1bcc288accd0b101c7b179f57c7b5c4bd7be.png) The account Owner and any service accounts always have full product capability, subject to the same plan gate. See [Service Accounts](https://testingbot.com/support/team/service-accounts). ## Built-in roles The three built-in roles, **Admin** , **User** , and **Viewer** , all include access to every product your plan provides. The difference between them is not which products they can open, but what they can do inside those products: creating, running, and deleting tests. - **Admin** has full capability, including deleting tests. - **User** can view and run tests, but cannot delete them. - **Viewer** is read-only: view tests and reports, but cannot run or delete. For the complete capability breakdown, see [Roles & Permissions](https://testingbot.com/support/team/rbac/roles-and-permissions) and [Member Roles](https://testingbot.com/support/team/rbac/member-roles). ## Restrict to specific products To give a team access to only some products rather than all of them, create an Enterprise [custom role](https://testingbot.com/support/team/rbac/custom-roles) and enable only the product permissions you want. For example, a custom role could grant Visual Testing and Accessibility Testing while leaving Automated Testing disabled. Custom roles are RBAC-only: they control product and test capability, but they can never grant identity and access management (IAM) permissions such as managing billing or team members. Assign a custom role to a member from [Team Members](https://testingbot.com/members/sub) the same way you assign a built-in role. ## Available products The six products and their permission keys are listed below. Each key also requires your plan to include that product, shown by the **plan** badge in the role editor. - **Live Web Testing** : `product.live_web.access` - **Live App Testing** : `product.live_app.access` - **Automated Testing** : `product.automation.access` - **Visual Testing** : `product.visual.access` - **Accessibility Testing** : `product.accessibility.access` - **AI Testing** : `product.ai.access` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/team/account # How to update my account details? You can update your account details (email, first name, last name) by going to [Members \> Account](https://testingbot.com/members/user/edit). In the form "Edit your account details" you can update your personal details. ## Resetting Security Tokens You can find your Security Tokens (`Key` + `Secret`) by going to [Members \> Manage Credentials](https://testingbot.com/members/user/security). These two identifiers are necessary to authenticate to our Selenium grid. It is possible to reset these security tokens by clicking the "Generate new keys" link. **Caution:** resetting these tokens means your existing tests with older key/secret credentials will stop running. ## Change Password You can change your current password by logging into TestingBot and going to [Members \> Security](https://testingbot.com/members/user/security). Click "Change Password". You will be asked to fill in your current password and a new password. ## Active Sessions On the [Security page](https://testingbot.com/members/user/security) you can find an overview of the active sessions for your account. Each entry contains the logged-in IP address, device name and last active time. You can sign out of individual sessions, or sign out everywhere. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/team/audit-logs # TestingBot Audit Logs Audit logs provide a comprehensive record of all activities performed by team members in your TestingBot organization. These logs help you maintain security, ensure compliance and monitor user behavior across your testing infrastructure. Only organization administrators can view and manage audit logs. Audit logs are retained for 400 days. ## How to Access Audit Logs To view your organization's audit logs: 1. Log into your TestingBot account 2. Navigate to [Members → Team Activity](https://testingbot.com/members/sub/activity) 3. View the comprehensive activity log for your organization The audit log interface provides filtering options to help you find specific events by date range, event type or team member. ## Event Types TestingBot groups the tracked activities into the following categories: ### Authentication | Event | Description | | --- | --- | | Login | User successfully logged into the TestingBot platform | | Logout | User logged out of the TestingBot platform | | Failed Login | A failed sign-in attempt was recorded for an account | ### Account Management | Event | Description | | --- | --- | | Add Account | New team member was added to the organization | | Remove Account | Team member was removed from the organization | | Profile Update | User updated their profile information (name, email, etc.) | | Email Changed | A user changed their account email address (old and new address recorded) | | Role / Permission Changed | A team member's read-only (Viewer) permission was enabled or removed | ### Plan & Configuration | Event | Description | | --- | --- | | Upgrade Level | Account plan was upgraded to a higher tier | | Downgrade Level | Account plan was downgraded to a lower tier | | Max Parallel Changed | Maximum parallel test sessions limit was modified | | Max Parallel Device Changed | Maximum parallel device sessions limit was modified | ### Security | Event | Description | | --- | --- | | Password Changed | User changed their account password | | Change Keys | User regenerated their API access keys (Key + Secret) | | 2FA Enabled | Two-factor authentication was enabled for the account | | 2FA Disabled | Two-factor authentication was disabled for the account | | Sessions Cleared | An administrator force-cleared a team member's active sessions | | Audit Log Exported | A team administrator exported the audit log to CSV | ### Team Invitations | Event | Description | | --- | --- | | Invite Sent | Team invitation was sent to a new member | | Invite Accepted | Team member accepted their invitation and joined the organization | | Invite Denied | Team invitation was declined or rejected | | Invite Resent | A pending team invitation was resent to the recipient | | Invite Revoked | A pending team invitation was cancelled before being accepted | ### Integrations | Event | Description | | --- | --- | | Webhook Created | A webhook endpoint was created for the account | | Webhook Updated | An existing webhook endpoint was updated | | Webhook Deleted | A webhook endpoint was deleted | | Integration Connected | A third-party integration (Jira, Slack or Microsoft Teams) was connected | | Integration Disconnected | A third-party integration was disconnected | | Notification Settings Changed | Notification or SMS alert settings were updated | ### Billing | Event | Description | | --- | --- | | Subscription Cancelled | The account subscription was cancelled | | Subscription Paused | A subscription pause was requested | | Retention Discount Requested | A retention discount was requested from the cancellation flow | ## Filtering and Searching The audit log interface provides several filtering options to help you find specific events: - **Filter by Event Type:** Select specific event types to view only those activities - **Filter by User:** Show activities performed by a specific team member - **Filter by Date Range:** Specify a custom date range to narrow down the results ## Data Retention and Export You can export all audit logs in `.CSV` format. For compliance and record-keeping purposes, we recommend regularly exporting your audit logs if you need to maintain historical records beyond the 400-day retention period. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/team/sso # Setting Up Single Sign On (SSO) TestingBot supports Identity Provider (IdP)-initiated and Service Provider (SP)-initiated Single Sign-On (SSO) via the SAML 2.0 protocol. This allows your authorized employees to access TestingBot in a moderated fashion, as an alternative to using passwords. To start using SSO, please make sure you have an [Identity Provider (IdP)](https://testingbot.com#idp) set up. ## Setting Up Identity Provider TestingBot provides preconfigured SAML applications for a selection of Identity Providers (IdPs). These applications allow you to integrate SSO with TestingBot in a very easy way. Below is a list of SAML applications available: - [Microsoft Entra](https://testingbot.com/support/team/sso/microsoft-entra) - [Okta](https://testingbot.com/support/team/sso/okta) To set up a custom Identity Provider, such as Salesforce, ForgeRock, Auth0 or others, please follow the steps below. 1. Retrieve the [SAML metadata](https://testingbot.com/users/saml/metadata) from TestingBot. 2. Log in to your identity provider's administrator panel. 3. If your IdP does not allow you to upload the metadata file, you can set up the integration manually. 4. Export SAML metadata of your newly created SAML application. Please [email it to us](https://testingbot.com/contact/new) so that we can set up the connection for you. ## Service Provider SAML Requirements Below is a list of settings that are required by the TestingBot Service Provider: | Setting Name | Value | | --- | --- | | Entity ID | `https://testingbot.com/users/saml/metadata` | | Assertion Consumer Service (ACS URL, Reply URL) | `https://testingbot.com/users/saml/auth` | | Name ID (Unique User Identifier) | `email` | | Name ID format/policy | `urn:oasis:names:tc:SAML:2.0:nameid-format:transient` | | Binding | `urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST` | | Idp Cert Fingerprint Algorithm | `http://www.w3.org/2000/09/xmldsig#sha256` | | Login URL | `https://testingbot.com/users/saml/sign_in` | ### SAML Claims TestingBot Service Provider supports the following SAML custom claims: - `email` - `http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress` - `http://schemas.xmlsoap.org/ws/2005/05/identity/claims/email` - `first_name` - `http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname` - `last_name` - `http://schemas.xmlsoap.org/ws/2005/05/identity/claims/surname` ## SSO Options We provide the following options with Single Sign On: ### Just-In-Time (JIT) Provisioning If a user from your organization logs in through SSO, we can automatically create an account for this user on TestingBot, which will be associated to your team. Each user will be able to see the tests created by other members of your organization. If this setting is disabled, users that are not yet registered with TestingBot will not be able to use TestingBot. #### Enforce SSO (Big Bang) When this setting is enabled, users in your organization must log in through SSO. All other authentication methods will not be allowed. #### Email verification New users logging in through SAML for the first time will automatically have their account (and email address) verified. #### Logout TestingBot supports both SP-Initiated logout and IdP-initiated logout. If you'd like to set up SP-initiated logout, please [contact us](https://testingbot.com/contact/new) with your IdP SLO endpoint. ## Unsupported Features We currently do not provide support for these features: - SCIM (System for Cross-domain Identity Management) ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/team/sso/microsoft-entra # Setting up SSO with Microsoft Entra TestingBot has created a SAML app in Microsoft Entra which you can use to easily enable SSO with Entra and TestingBot. 1. Go to **Browse Microsoft Entra Gallery** and search for **TestingBot**. 2. Click the TestingBot app to go to the detail page. Click the **Create** button to add the app. 3. In the **Properties** section, make sure the **User Identifier** claim is set to a valid email address. You can adjust this later under **Attributes & Claims** in the Entra SAML configuration. 4. Click **Single Sign on** in the Manage section, then select **SAML** as the single sign-on method. 5. Make sure you fill in the required information in the **Basic SAML Configuration** , such as the **Identifier (Entity ID)** and **Reply URL (Assertion Consumer Service URL)** as listed on the [SSO setup page](https://testingbot.com/support/team/sso). 6. Download the **Federation Metadata XML** from the SAML Signing Certificate section, then send it to TestingBot via the [contact form](https://testingbot.com/contact/new) so we can finish setting up SSO on our side. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/team/sso/okta # Setting up SSO in Okta TestingBot has created a SAML app in Okta which you can install to easily enable SSO with Okta and TestingBot. 1. Log in to the **Okta administrator panel** , go to **Applications** and search for TestingBot in the **Browse App Integration Catalog**. 2. Click the TestingBot app to go to the detail page. Click the **Add Integration** button to add the TestingBot app. 3. Change the name of the application to your liking, or leave it as TestingBot, then click **Done**. 4. You can now assign one or more users or groups to use the TestingBot application. 5. Click the **Sign On** tab. 6. In the **Credentials Details** section, make sure that the value for **Application username format** is a valid email address. The default format for Okta username is an email address, unless it has been changed in the Okta Admin Console. 7. In the **SAML Signing Certificates** section, click **Actions** , and then **View IdP metadata**. Please [send the metadata to TestingBot](https://testingbot.com/contact/new). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/team/service-accounts # Service Accounts A service account is an API-only identity that runs tests with its own key and secret. Unlike a [sub-account](https://testingbot.com/support/team/sub-accounts), a service account has no dashboard login: it exists only to authenticate automated tests against the TestingBot REST API and the browsers + device grid. Every service account belongs to your account owner. Its tests run on your subscription, share your concurrency and appear under your account. It acts exactly like the tests of any team member. You manage service accounts from [Team Management \> Service Accounts](https://testingbot.com/members/service_accounts) (team admins only). - Each service account has its own **TestingBot Key** and **TestingBot Secret**. - A service account **cannot sign in** to the TestingBot dashboard and is not counted as a dashboard user. - Its credentials can be rotated or revoked on their own, without affecting anyone's personal key/secret. ## When to Use a Service Account Service accounts are the recommended way to run tests from shared, non-human contexts. Use one when you want to: - Run tests from [CI/CD pipelines](https://testingbot.com/support/integrations/ci-cd) without tying them to a person's login. - Give each pipeline, environment, or team its own labelled credentials so you can tell their runs apart. - Rotate or revoke a set of credentials independently if they leak or a pipeline is retired, without disrupting your own access. ## Creating a Service Account Go to [Team Management \> Service Accounts](https://testingbot.com/members/service_accounts) and click **Add Account**. Enter an [identifier](https://testingbot.com#identifier) (at least 3 characters) and click **Submit**. The new service account appears in the table straight away with its generated **TestingBot Key** and **TestingBot Secret**. Use the eye icon to reveal a value and the copy icon to copy it. You can come back and reveal these credentials at any time, so there is no one-time "save it now" step. ## The Identifier The identifier is a human-friendly label you give the service account, for example `ci-runner` or `nightly-regression`. It is used to mark the tests run by that service account on your dashboard and on the test detail page, so you can see at a glance which pipeline or environment started a test. The identifier must be at least 3 characters and must be unique within your account. You can change it later at any time. ## Using the Credentials Use the service account's key and secret anywhere you would normally use your own TestingBot key and secret. ### Browsers + Device Grid Point your tests at the TestingBot hub using the service account's key and secret: ```bash https://YOUR_KEY:YOUR_SECRET@hub.testingbot.com/wd/hub ``` Or pass them in your capabilities (`tb:options` / `key` and `secret`) exactly as you would with a personal key/secret. ### REST API The [REST API](https://testingbot.com/support/api) uses HTTP Basic authentication with the key and secret: ```bash curl -u YOUR_KEY:YOUR_SECRET https://api.testingbot.com/v1/tests ``` We recommend storing the key and secret as environment variables or CI secrets, never committing them to source control. ## Editing a Service Account To rename a service account, open the actions menu (the **...** button) next to it and choose **Edit**. Change the identifier and click **Submit**. Editing the identifier does not change the key or secret. ## Rotating the Access Key From the actions menu, choose **Rotate access key** to generate a fresh key and secret for the service account. This is useful if a credential may have leaked. **Rotating takes effect immediately:** the previous key and secret stop working right away, so any automation still using the old credentials will start failing to authenticate. Update your pipelines with the new key and secret after rotating. ## Deleting a Service Account From the actions menu, choose **Delete** to remove a service account. Its key and secret stop working immediately. Tests that were already run by the service account stay in your account and keep their history; they remain labelled with the identifier on the test detail page. ## How Test Runs Are Attributed Tests started by a service account run on your account owner's subscription: they consume your plan's concurrency and minutes and are visible to your team, just like any other test. On the test detail page, the **Started by** field shows the service account's identifier so you can tell its runs apart from those of human team members. ## Service Accounts vs Sub-Accounts Both run tests under your account, but they serve different purposes: - **Sub-account:** a person on your team. It has a dashboard login, can view and manage tests, and can be given roles (Admin / Member / Viewer). See [Team & Sub-Accounts](https://testingbot.com/support/team/sub-accounts). - **Service account:** an API-only identity for automation. It has no dashboard login, no role management, and no 2FA. It exists purely to run tests with a dedicated, rotatable key and secret. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/other # Other Documentation Reference pages for the TestingBot Selenium and Appium grid — configuration details, error code lookups, CI/CD build status widgets, proxy setup and miscellaneous integrations. ## Grid reference - [Grid configuration](https://testingbot.com/support/other/configuration) — endpoints, regions and infrastructure overview. - [Common errors and failures](https://testingbot.com/support/other/errors) — error code reference for Selenium and Appium tests. - [Selenium WebDriver examples](https://testingbot.com/support/other/selenium2) — example scripts in Ruby, Python, PHP, Java and C#. ## CI/CD widgets - [Status badges](https://testingbot.com/support/other/status-badges) — embed Selenium/Appium build status badges in GitHub README files. - [Browser matrix widgets](https://testingbot.com/support/other/browser-matrix) — embed a live browser-coverage matrix from your test runs. - [Share test results](https://testingbot.com/support/other/sharing) — view, share and embed Selenium test recordings. ## Integrations & advanced setup - [Browsermob proxy / HAR files](https://testingbot.com/support/other/proxy) — capture network traffic during Selenium tests. - [TestingBot on GitHub](https://testingbot.com/support/other/github) — open-source client libraries and example projects. - [Sikuli image-based testing](https://testingbot.com/support/other/sikuli) — image-recognition testing alongside Selenium. - [Firefox Unbranded for extensions](https://testingbot.com/support/other/firefox-unbranded) — install and test signed Firefox add-ons. ## Related - [TestingBot Tunnel](https://testingbot.com/support/tunnel) — test internal websites and pre-production environments. - [CI/CD integrations](https://testingbot.com/support/integrations/ci-cd) — Jenkins, GitHub Actions, GitLab CI, CircleCI and more. - [Selenium capabilities reference](https://testingbot.com/support/web-automate/selenium/test-options) — all tb:options capabilities. --- URL: https://testingbot.com/support/other/configuration # TestingBot Grid Configuration TestingBot provides a distributed cloud grid of real browsers and devices. Connect your Selenium, Appium, Puppeteer, Playwright, Cypress, XCUITest and Espresso tests to this grid through the endpoints documented below. The Selenium and Appium grid runs the latest stable Selenium 4 server and is fully W3C-compliant. Your client SDK version (Selenium, Appium, Playwright, Puppeteer) is matched on the grid side for compatibility. ## Service status Live incidents and scheduled maintenance are posted on the [TestingBot status page](https://status.testingbot.com/) (RSS feed available). Subscribe to the status page to receive notifications for grid availability, hub-endpoint issues and tunnel infrastructure events. [![Uptime report for testingbot.com: last 30 days](https://share.pingdom.com/banners/3eeaadba "Uptime report for testingbot.com: last 30 days")](https://stats.pingdom.com/kwiiczwo3ltc) [![Uptime report for Selenium grid: last 30 days](https://share.pingdom.com/banners/05c747d9 "Uptime report for Selenium grid: last 30 days")](https://stats.pingdom.com/kwiiczwo3ltc) ## TestingBot IP Ranges Below you will find an up-to-date list of the TestingBot IP ranges used during testing. - 109.68.162.161 - 109.68.162.165 - 78.20.187.211 - 162.55.135.22 - 95.217.83.184 - 95.216.39.110 - 95.216.229.109 - 142.132.137.7 - 195.201.167.43 - 162.55.24.100 - 185.223.133.13 - 135.181.118.186 - 188.34.142.238 - 95.217.228.116 - 65.109.76.224 - 176.9.121.211 - 194.78.229.67 - 87.64.203.47 You can also fetch this up-to-date list from the [TestingBot REST-API](https://testingbot.com/support/api/devices#ipranges). ### Geolocation IP ranges TestingBot provides [geolocation testing](https://testingbot.com/support/web-automate/selenium/test-options#geo), allowing you to test from various geographical places around the world. Below are the IP addresses we use for this. - **AU** Australia: `35.213.216.75` - **BE** Belgium: `109.68.162.161` - **BR** Brazil: `35.215.244.91` - **CA** Canada: `34.47.9.124` - **CL** Chile: `34.176.132.166` - **FR** France: `15.236.216.234` - **DE** Germany: `188.245.92.209` - **IN** India: `35.207.203.231` - **IT** Italy: `15.161.57.62` - **JP** Japan: `35.213.44.184` - **NO** Norway: `51.120.9.213` - **SG** Singapore: `18.138.255.42` - **ZA** South Africa: `13.245.163.253` - **SE** Sweden: `16.171.19.152` - **CH** Switzerland: `34.65.52.54` - **AE** United Arab Emirates: `20.174.25.155` - **GB** United Kingdom: `35.214.33.251` - **US** United States: `178.156.132.19` ## Selenium grid configuration W3C-compliant Selenium 4 endpoint. Connect your Selenium WebDriver client (Java, Python, Ruby, Node.js, C#, PHP) to the URL below and pass your API key and secret inside the `tb:options` capability map. | Key | Value | | --- | --- | | Selenium version | Latest stable Selenium 4 (W3C protocol). The grid is fully W3C-compliant. [Pin a specific Selenium version](https://testingbot.com/support/web-automate/selenium/test-options#seleniumversion) with the seleniumVersion capability if needed. | | Hub endpoint | `https://hub.testingbot.com/wd/hub` | | Port | 443 (HTTPS, recommended). Plain HTTP on port 80 and 4444 is also accepted but not recommended for production. | | Default idle timeout | 130 seconds. [Change the idleTimeout capability](https://testingbot.com/support/web-automate/selenium/test-options#idletimeout) to raise or lower this per test. | | Default screen resolution | 1280x1024. [Change the screenResolution capability](https://testingbot.com/support/web-automate/selenium/test-options#screenresolution) to pick a different resolution per test. | ## Appium grid configuration Appium 2 endpoint for native mobile app testing and mobile-browser testing on iOS and Android simulators, emulators and real devices. Uses the same hub URL as Selenium. | Key | Value | | --- | --- | | Appium version | Latest compatible Appium version for the specified device, OS and OS version. [Pin a specific Appium version](https://testingbot.com/support/app-automate/appium/options#appiumVersion) with the appiumVersion capability. | | Hub endpoint | `https://hub.testingbot.com/wd/hub` | | Port | 443 (HTTPS, recommended). Plain HTTP on port 80 and 4444 is also accepted but not recommended for production. | | Default idle timeout | 130 seconds. [Change the idleTimeout capability](https://testingbot.com/support/web-automate/selenium/test-options#idletimeout) per test if needed. | ## Puppeteer and Playwright configuration Puppeteer and Playwright connect over a WebSocket endpoint rather than the WebDriver HTTP hub. Use the appropriate `wss://` URL below and pass your API key, secret, browserName and browserVersion as query parameters or capabilities depending on your client. See [the Playwright support page](https://testingbot.com/support/web-automate/playwright) and [the Puppeteer support page](https://testingbot.com/support/web-automate/puppeteer) for complete examples. | Key | Value | | --- | --- | | Playwright endpoint | `wss://cloud.testingbot.com/playwright` | | Puppeteer endpoint | `wss://cloud.testingbot.com/puppeteer` | | Playwright or Puppeteer version | TestingBot matches the server-side version to the client-side version you use, ensuring full compatibility. No version pin is needed on the capabilities. | | Port | 443 (TLS-encrypted WebSocket). Plain ws:// on port 80 is also accepted but not recommended for production. | ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/other/sharing ### Available Topics: - [Link to a Test](https://testingbot.com#linktest) - [Link to a Build](https://testingbot.com#linkbuild) - [Embed Test Info](https://testingbot.com#embedinfo) - [Embed Test Video](https://testingbot.com#embedvideo) # Sharing Tests Below are various ways to share, embed or link to a test or build on TestingBot. To share, view or link to a test, it is important to know its `sessionId`. Selenium and Appium will return a `sessionId` when a client starts a new session. This sesssionId is a unique identifier and changes per test. It is stored as a member variable of the instantiated Driver object. Most of the time it is named `sessionId` or `session_id`, depending on the client you are using. ## Link to a Test You can easily share a single test result with someone in a safe way, without the other person needing an account. To share a test result, you'll need to create the following url: https://testingbot.com/tests/ **sessionId**?auth= **authentication-hash** The authentication hash is created with md5 in the following format: [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby Digest::MD5.hexdigest("#{client_key}:#{client_secret}:#{session_id}") ``` ```java MessageDigest m=MessageDigest.getInstance("MD5"); String s = "TESTINGBOT_KEY:TESTINGBOT_SECRET:session_id"; m.update(s.getBytes(),0,s.length()); System.out.println("MD5: "+new BigInteger(1,m.digest()).toString(16)); ``` ```php echo md5("TESTINGBOT_KEY:TESTINGBOT_SECRET:$session_id"); ``` ```python import hashlib print(hashlib.md5("TESTINGBOT_KEY:TESTINGBOT_SECRET:#{session_id}".encode('utf-8')).hexdigest()) ``` ```javascript var crypto = require('crypto'); crypto.createHash('md5').update("TESTINGBOT_KEY:TESTINGBOT_SECRET:sessionId").digest("hex"); ``` ```csharp Convert.ToBase64String(new System.Security.Cryptography.MD5CryptoServiceProvider().ComputeHash(System.Text.Encoding.UTF8.GetBytes("TESTINGBOT_KEY:TESTINGBOT_SECRET:sessionId"))); ``` An example link would then be: https://testingbot.com/tests/9jf392fj3j2f3ojid8?auth=230i439uf3fojf ## Link to a Video You can easily share a video of a specific test, please see the example below: To build the URL with the correct authentication hash, the URL should be structured like this: https://testingbot.com/tests/ **sessionId.mp4**?auth= **authentication-hash** The authentication hash is created with MD5 in the following format: [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby Digest::MD5.hexdigest("#{client_key}:#{client_secret}:#{session_id}") ``` ```java MessageDigest m=MessageDigest.getInstance("MD5"); String s = "TESTINGBOT_KEY:TESTINGBOT_SECRET:session_id"; m.update(s.getBytes(),0,s.length()); System.out.println("MD5: "+new BigInteger(1,m.digest()).toString(16)); ``` ```php echo md5("TESTINGBOT_KEY:TESTINGBOT_SECRET:$session_id"); ``` ```python import hashlib print(hashlib.md5("TESTINGBOT_KEY:TESTINGBOT_SECRET:#{session_id}".encode('utf-8')).hexdigest()) ``` ```javascript var crypto = require('crypto'); crypto.createHash('md5').update("TESTINGBOT_KEY:TESTINGBOT_SECRET:sessionId").digest("hex"); ``` ```csharp Convert.ToBase64String(new System.Security.Cryptography.MD5CryptoServiceProvider().ComputeHash(System.Text.Encoding.UTF8.GetBytes("TESTINGBOT_KEY:TESTINGBOT_SECRET:sessionId"))); ``` An example link would then be: https://testingbot.com/tests/9jf392fj3j2f3ojid8.mp4?auth=230i439uf3fojf ## Link to a Build With TestingBot, it is possible to group several tests under one build identifier. In our member area, you can then see all the tests grouped by builds. The build overview at `https://testingbot.com/builds/YOUR_TESTINGBOT_KEY` is only available for accounts whose test results are public. For a private account, share a link to a single build instead. To link directly to a build, please see this example: `https://testingbot.com/builds/YOUR_TESTINGBOT_KEY/build-identifier?auth=authentication-hash` To generate the authentication-hash, you must concatenate user key, user secret and build-identifier, then generate the MD5 hash: [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby Digest::MD5.hexdigest("#{client_key}:#{client_secret}:#{build_identifier}") ``` ```java MessageDigest m=MessageDigest.getInstance("MD5"); String s = "TESTINGBOT_KEY:TESTINGBOT_SECRET:build_identifier"; m.update(s.getBytes(),0,s.length()); System.out.println("MD5: "+new BigInteger(1,m.digest()).toString(16)); ``` ```php echo md5("TESTINGBOT_KEY:TESTINGBOT_SECRET:$build_identifier"); ``` ```python import hashlib print(hashlib.md5("TESTINGBOT_KEY:TESTINGBOT_SECRET:#{build_identifier}".encode('utf-8')).hexdigest()) ``` ```javascript var crypto = require('crypto'); crypto.createHash('md5').update("TESTINGBOT_KEY:TESTINGBOT_SECRET:build_identifier").digest("hex"); ``` ```csharp Convert.ToBase64String(new System.Security.Cryptography.MD5CryptoServiceProvider().ComputeHash(System.Text.Encoding.UTF8.GetBytes("TESTINGBOT_KEY:TESTINGBOT_SECRET:build_identifier"))); ``` The build-identifier is the custom string you can [send as an option to group several tests.](https://testingbot.com/support/web-automate/selenium/test-options#build) ## Embed Test Info You can embed a test page in any CI detail page, test report or custom website. Please see the example below. You add a SCRIPT tag in your page which links to the test. Replace sessionId with the correct sessionId of the test. ```php ``` To generate the authentication-token, you must concatenate user key, user secret and sessionId, then generate the MD5 hash: [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby Digest::MD5.hexdigest("#{client_key}:#{client_secret}:#{sessionId}") ``` ```java MessageDigest m=MessageDigest.getInstance("MD5"); String s = "TESTINGBOT_KEY:TESTINGBOT_SECRET:sessionId"; m.update(s.getBytes(),0,s.length()); System.out.println("MD5: "+new BigInteger(1,m.digest()).toString(16)); ``` ```php echo md5("TESTINGBOT_KEY:TESTINGBOT_SECRET:$sessionId"); ``` ```python import hashlib print(hashlib.md5("TESTINGBOT_KEY:TESTINGBOT_SECRET:#{sessionId}".encode('utf-8')).hexdigest()) ``` ```javascript var crypto = require('crypto'); crypto.createHash('md5').update("TESTINGBOT_KEY:TESTINGBOT_SECRET:sessionId").digest("hex"); ``` ```csharp Convert.ToBase64String(new System.Security.Cryptography.MD5CryptoServiceProvider().ComputeHash(System.Text.Encoding.UTF8.GetBytes("TESTINGBOT_KEY:TESTINGBOT_SECRET:sessionId"))); ``` ## Embed Video Info Next to embedding test info pages, you can also embed the video of the test. This might be useful to include in test reports or on CI result pages. ```php ``` To generate the authentication-token, you must concatenate user key, user secret and sessionId, then generate the MD5 hash: [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby Digest::MD5.hexdigest("#{client_key}:#{client_secret}:#{sessionId}") ``` ```java MessageDigest m=MessageDigest.getInstance("MD5"); String s = "TESTINGBOT_KEY:TESTINGBOT_SECRET:sessionId"; m.update(s.getBytes(),0,s.length()); System.out.println("MD5: "+new BigInteger(1,m.digest()).toString(16)); ``` ```php echo md5("TESTINGBOT_KEY:TESTINGBOT_SECRET:$sessionId"); ``` ```python import hashlib print(hashlib.md5("TESTINGBOT_KEY:TESTINGBOT_SECRET:#{sessionId}".encode('utf-8')).hexdigest()) ``` ```javascript var crypto = require('crypto'); crypto.createHash('md5').update("TESTINGBOT_KEY:TESTINGBOT_SECRET:sessionId").digest("hex"); ``` ```csharp Convert.ToBase64String(new System.Security.Cryptography.MD5CryptoServiceProvider().ComputeHash(System.Text.Encoding.UTF8.GetBytes("TESTINGBOT_KEY:TESTINGBOT_SECRET:sessionId"))); ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/other/sikuli # Sikuli Cloud Testing Sikuli uses image recognition to identify components you want to interact with on the screen. While Selenium is limited to interacting with browsers, Sikuli can be used to interact with native GUI components (for example: Windows dialogs, Linux programs or OS-X GUI). At TestingBot, all our VMs are able to run your Sikuli scripts. ## 1. Download and Setup Sikuli To get started, please [download Sikuli](http://www.sikulix.com). Once downloaded, run the file to install Sikuli. ## 2. Create a new Sikuli script ![Sikuli script](https://testingbot.com/assets/support/sikuli_script-60b7ed26c56bb2d57ddeab1c5820dee0a5644a4b18f3b862b156ae8b0f7084c8.png) Sikuli uses image-recognition to find the components to interact with. In this example, we'll take a screenshot of a Chrome dialog asking to install a Chrome extension. ## 3. The Sikuli script As you can see on the screenshot above, you can use `wait([image], FOREVER)` to have Sikuli wait for something to appear on the screen that looks like the `[image]`. Once this happens, Sikuli will move on to the next step in the script: clicking the component it detected. ## 4. Integration with TestingBot At TestingBot, we can run your Sikuli script at the start of your Selenium WebDriver test. Simply create a zip-file with one or more of your `.sikuli` scripts inside. Then use this URL in your `DesiredCapabilities` when starting a test. This way you can run Sikuli tests in the cloud on all our [platforms](https://testingbot.com/support/web-automate/browsers). For example: ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("platform", "VISTA"); caps.setCapability("version", "9"); caps.setCapability("browserName", "internet explorer"); caps.setCapability("sikuli", "http://mywebsite.com/sikuli-scripts.zip"); ``` This will make sure TestingBot will download your Sikuli script and run it at the start of your test. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/other/errors # TestingBot Test Errors and How to Fix Them Reference for the most common errors TestingBot returns when an automated or live test ends unexpectedly. Each section explains what triggered the error, the most common causes, and how to fix it. Errors appear on the test detail page in the dashboard and in the REST API response. ## Minutes depleted ### What it means The test failed because your account does not have enough test minutes left. New tests cannot be started on TestingBot until minutes are restored. ### How to resolve See [our pricing plans](https://testingbot.com/pricing) and upgrade your subscription to add minutes immediately, or wait for your monthly plan to reset. Free-tier accounts receive a fresh allowance each month. Enterprise plans can request additional minute pools mid-cycle. ## Test exceeded max duration ### What it means The test was terminated because it exceeded the maximum duration allowed. The default cap is 1800 seconds (30 minutes). ### Common causes - The test is genuinely long. Consider splitting it into multiple smaller, atomic tests. - Tests on the cloud can be slower than local runs because of network round-trips and remote VM startup. - The test is stuck in an endless loop, repeatedly sending commands. ### How to resolve - Break the test into smaller pieces. - Use the [maxDuration capability](https://testingbot.com/support/web-automate/selenium/test-options#maxduration) to raise the cap if you legitimately need a longer session. - Check for endless loops and add explicit timeouts to waits. ## Test idle timeout ### What it means TestingBot did not receive a new command from your Selenium or Appium script for more than 90 seconds (the default idle timeout). The session is terminated to prevent it from running forever and consuming all your minutes. ### Common causes - A missing `driver.quit()` at the end of the test. The runner finishes but the session keeps the browser alive. - The test script crashed, was interrupted, or lost internet connectivity. - A legitimately slow step that takes more than 90 seconds between commands. ### How to resolve - Ensure the test calls `driver.quit()` in a teardown or `finally` block so it always runs. - Run the test from a stable network. CI runners with unstable egress are a common cause. - Raise the timeout with the [idleTimeout capability](https://testingbot.com/support/web-automate/selenium/test-options#idletimeout) if you need more than 90 seconds between commands. ## New session cancelled before test start ### What it means Your test runner requested a new session, but it closed the connection before TestingBot could allocate a slot and return a session id. ### Common causes - Client connection timeout too short. iOS and Android simulators can take 60 to 120 seconds to boot. - Running more tests than your plan's concurrency limit. Queued tests still consume the runner's connection time. - Grid-side capacity issues during peak times. Check the [TestingBot status page](https://status.testingbot.com/). ### How to resolve - Raise the connection timeout in your test runner. For WebdriverIO use `connectionRetryTimeout: 210000` (3.5 minutes). For Selenium HttpClient set the equivalent socket timeout. - Confirm you are within the concurrent-session limit on your plan. - Subscribe to the [status page RSS feed](https://status.testingbot.com/) for incident notifications. ## Manual session ended due to inactivity ### What it means A Live (interactive) browser or device session was terminated because there was no mouse or keyboard input for an extended period. Live sessions are billed by the minute, so idle sessions are ended automatically to prevent runaway charges. ### How to resolve Start a new Live session when you are ready to continue. If you need a long-running pinned session for QA workflows, consider switching to an automated test with a deliberate `maxDuration` setting. ## Capability not found or browser unavailable ### What it means You will see a `CapabilityNotPresentOnTheGridException` (or a similar "browser not found" message) when the browser, version and platform combination you requested is not currently offered on the TestingBot grid. ### How to resolve - Check the live [supported browser list](https://testingbot.com/support/web-automate/browsers) for available combinations. - Use the version aliases `latest`, `latest-1`, `latest-2` when you do not need to pin to an exact version. TestingBot resolves them to the current top release. - Double-check the spelling and case of `platformName` values. Selenium 4 W3C uses `WINDOWS`, `macOS`, `LINUX`, etc. ## Authentication errors (401) ### What it means Your API key and secret were missing, malformed, or did not match the credentials on file. The grid rejects the session before any browser is allocated. ### How to resolve - Verify your key and secret in [your account settings](https://testingbot.com/members/user/edit). - Pass credentials inside `tb:options` (W3C or Selenium 4), not as top-level capabilities. - If you have rotated keys, update every CI secret store that references the old pair. ## More help Status incidents are posted at the [TestingBot status page](https://status.testingbot.com/) (RSS feed available). For anything not covered above, [contact our support team](https://testingbot.com/contact/new) with your session ID. The dashboard URL of the failed test is the fastest way for us to investigate. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/other/test-speed-optimization # Test Speed Optimization Every automated test session on TestingBot goes through the same phases: the session is requested, waits for a free slot if your account is at its concurrency limit, a browser or device is prepared (setup time), your commands run (execution time) and the session completes. This guide explains where that time goes and how to reduce it. ## See the duration breakdown for your tests On every test detail page in the [member dashboard](https://testingbot.com/members/tests), hover over the **Duration** value to see a breakdown of the session: how much time was spent on setup versus execution, and which capabilities impacted the session speed. Each capability in the popover links to the matching section on this page. ## Optimize TestingBot Time TestingBot Time is everything between requesting a session and its completion. It is the sum of two parts: - **Setup time** : queue time plus preparing the browser, emulator, simulator or physical device, before your first command runs. - **Execution time** : the time your test commands spend running on the browser or device. ### Reduce setup time - **Stay within your concurrency limit.** Sessions started beyond your plan's parallel limit wait in a queue until a slot frees up. Spread test starts over time, or upgrade the number of parallel sessions if your suite regularly queues. - **Prefer virtual devices for broad coverage.** Android emulators and iOS simulators are prepared faster than physical devices. Use virtual devices for functional coverage and reserve physical devices for final verification. - **Only set the session capabilities you need.** Capabilities such as [timeZone](https://testingbot.com#timezone), [screen-resolution](https://testingbot.com#screen-resolution), [prerun](https://testingbot.com#prerun) and [firefox\_profile](https://testingbot.com#firefox-profile) each add work before the session is ready. The [capability sections below](https://testingbot.com#optimize-capabilities-impacting-session) explain the cost of each one. - **Reuse uploaded apps.** For mobile app tests, upload your app once and reference it with its `tb://` app key instead of uploading the binary again for every session. - **Set a realistic client connection timeout.** iOS simulators and Android emulators can take 60 to 120 seconds to boot. If your test runner gives up earlier, the session is cancelled and retried, which wastes the whole setup. See [new session cancelled before test start](https://testingbot.com/support/other/errors#session-cancelled). ### Reduce execution time - **Use explicit waits instead of fixed sleeps.** A `sleep(5)` always costs 5 seconds; an explicit wait continues the moment the element is ready. - **Keep tests atomic and run them in parallel.** Many short, independent tests finish faster than one long journey, because they spread across your parallel sessions and a failure does not invalidate unrelated steps. - **Quit the session as soon as the test ends.** Always call `driver.quit()` in a teardown block. A session without it stays open until the [idle timeout](https://testingbot.com/support/web-automate/selenium/test-options#idletimeout) hits, adding up to 90 seconds per test. - **Prefer fast selectors.** ID and CSS selectors are resolved faster than complex XPath expressions, especially on large pages. - **Report status through the API.** Use [test annotations](https://testingbot.com/support/web-automate/selenium/annotating-tests) or the REST API to mark a test passed or failed instead of adding extra in-browser steps for reporting. - **Avoid large downloads and uploads during the test.** Every file transferred through the browser session adds time. Keep fixtures small and mock heavy network responses where possible. ## Optimize Capabilities Impacting Session Some capabilities add processing while the session is being prepared or while your commands run. The duration breakdown popover on the test detail page flags the ones detected in your session with a **high** , **medium** or **low** impact badge. Each section below explains the cost and how to reduce it. ### screenrecorder (high impact) Video recording is enabled by default for every session. Encoding the video runs for the entire duration of the test and finalizing the file adds time when the session completes. If you do not need a recording, for example on large regression suites or consistently green builds, disable it with [`screenrecorder: false`](https://testingbot.com/support/web-automate/selenium/test-options#screenrecorder). ### tunnel (high impact) With [TestingBot Tunnel](https://testingbot.com/support/tunnel), browser traffic is routed through the secure tunnel to your machine, so every request pays extra network latency. Start the tunnel once and share it across all tests in the run instead of starting one per test, run the tunnel process close to the servers under test, and only enable it for tests that genuinely need access to internal or staging environments. ### geoCountryCode (high impact) [`geoCountryCode`](https://testingbot.com/support/web-automate/selenium/test-options#geo) routes all browser traffic through a proxy server in the selected country. Each request travels an extra round trip to that proxy. Only use it for tests that verify geolocation-specific behavior, and keep the rest of your suite on a direct connection. ### prerun (high impact) A [`prerun`](https://testingbot.com/support/web-automate/selenium/test-options#prerun) executable is downloaded and executed on the test VM before your session starts, so its full runtime is added to setup time. Keep prerun scripts small and fast, and prefer building one-time configuration into the app or the test itself. ### firefox\_profile (high impact) A custom [Firefox profile](https://testingbot.com/support/web-automate/selenium/firefox-profile) is uploaded and extracted on the VM during setup. The larger the profile, the longer the session takes to start. Keep the profile minimal: only the preferences and extensions the test actually needs. ### screenshot (medium impact) With [`screenshot: true`](https://testingbot.com/support/web-automate/selenium/test-options#screenshot), TestingBot captures a screenshot during the test, which adds overhead per captured step. For large suites, disable automatic screenshots and take targeted screenshots from your test code only where they add debugging value. ### timeZone (medium impact) Setting a custom [`timeZone`](https://testingbot.com/support/web-automate/selenium/test-options#timezones) reconfigures the operating system clock before the session starts. Only set it for tests that assert timezone-dependent behavior. ### deviceOrientation (medium impact) Starting a mobile session in landscape requires rotating the device or emulator during setup. Use the default portrait orientation unless the test specifically covers landscape layouts, and rotate from within the test only when needed. ### realDevice (medium impact) Physical devices are cleaned and prepared between sessions, which takes longer than booting an emulator or simulator. Use virtual devices for day-to-day functional coverage and schedule physical-device runs where real hardware matters: sensors, performance characteristics and vendor-specific behavior. ### screen-resolution (low impact) Changing the [`screen-resolution`](https://testingbot.com/support/web-automate/selenium/test-options#screenresolution) switches the display mode of the VM during setup. Use the default resolution unless your test verifies a specific viewport, and prefer resizing the browser window from within the test when that is all you need. ### locale (low impact) Custom `locale` or `language` values are configured on the device or emulator during setup. Only set them for localization tests. --- URL: https://testingbot.com/support/other/status-badges # Status Badges Status Badges help you keep track of the status of your latest test build. You can simply embed markdown code or HTML code to your project website to show the status of your most recent test build. ## Builds In order to show a status badge, you need to make sure that you're running your tests in a group, called a **build**. For example: you have a project that has a bunch of tests. After every commit you want to make sure the tests still pass. The tests you want to run are grouped in a **build**. To make sure your tests are grouped in the same build, use Selenium's Desired Capabilities. Simple pass `"build" : "build-1XXX"` in your Desired Capabilities: ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("platform", "WIN7"); caps.setCapability("browserName", "Chrome"); caps.setCapability("build", "build-1XXX"); // for example ``` In our [member area](https://testingbot.com/members) you will then see a new build, `build-1XXX`, which will contain all the tests that ran with the same build identifier. ## Marking a build as passed/failed To mark a build as passed or failed, your test runner should be configured to send the success state of each test to TestingBot via the [TestingBot API](https://testingbot.com/support/api). - If for every test in the build we know that the test succeeded, then we'll automatically mark the build as **Passing**. - If a test failed, the build will automatically be marked as **Failed**. - If you don't send us the success state for every test in the build, then we can not know the success state of the build. In this case we show an **Unknown** state. ## Status Badge With our Status Badges, you can easily monitor the status of your most recent build. The `auth` parameter is generated by concatenating your `api key` and `api secret` with md5. Below are the different states that your badge can have: Badge | Status || | **Passing** | | | **Failure** | | | **Unknown** We don't know if this build passed or failed. Please use our [REST-API](https://testingbot.com/support/api) to send the success state of your individual tests to TestingBot. | | | **Error** The link to the badge might be invalid, or you might not have any builds yet. | **Markdown:** ```markdown [![TestingBot Test Status](https://testingbot.com/buildstatus/YOUR_TESTINGBOT_KEY?auth={authentication})](https://testingbot.com/members/builds) ``` **HTML:** ```html TestingBot Build Status ``` [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby Digest::MD5.hexdigest("#{client_key}:#{client_secret}") ``` ```java MessageDigest m=MessageDigest.getInstance("MD5"); String s = "TESTINGBOT_KEY:TESTINGBOT_SECRET"; m.update(s.getBytes(),0,s.length()); System.out.println("MD5: "+new BigInteger(1,m.digest()).toString(16)); ``` ```php echo md5("TESTINGBOT_KEY:TESTINGBOT_SECRET"); ``` ```python import hashlib print(hashlib.md5("TESTINGBOT_KEY:TESTINGBOT_SECRET".encode('utf-8')).hexdigest()) ``` ```javascript var crypto = require('crypto'); crypto.createHash('md5').update("TESTINGBOT_KEY:TESTINGBOT_SECRET").digest("hex"); ``` ```csharp Convert.ToBase64String(new System.Security.Cryptography.MD5CryptoServiceProvider().ComputeHash(System.Text.Encoding.UTF8.GetBytes("TESTINGBOT_KEY:TESTINGBOT_SECRET"))); ``` ## Private Accounts To display badges, you need to generate an **authentication token**. This token only unlocks the badge and build matrix images; it does not give access to your builds or tests, so the badge links to your builds in the member area. To generate the authentication token, you must concatenate your TestingBot key, TestingBot secret and optionally an identifier. Finally, take the MD5 hash of the concatenated string: [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) ```ruby Digest::MD5.hexdigest("#{client_key}:#{client_secret}") ``` ```java MessageDigest m=MessageDigest.getInstance("MD5"); String s = "TESTINGBOT_KEY:TESTINGBOT_SECRET"; m.update(s.getBytes(),0,s.length()); System.out.println("MD5: "+new BigInteger(1,m.digest()).toString(16)); ``` ```php echo md5("TESTINGBOT_KEY:TESTINGBOT_SECRET"); ``` ```python import hashlib print(hashlib.md5("TESTINGBOT_KEY:TESTINGBOT_SECRET".encode('utf-8')).hexdigest()) ``` ```javascript var crypto = require('crypto'); crypto.createHash('md5').update("TESTINGBOT_KEY:TESTINGBOT_SECRET").digest("hex"); ``` ## Open Source Accounts [Open Source](https://testingbot.com/open-source) accounts automatically have their tests and builds set to public. This means no authentication (auth parameter) is required to display badges or link to tests/builds. ## More - If you like to show status-badges of **multiple projects** , please consider creating a new user account per project. - We're also providing [browser-matrix images](https://testingbot.com/support/other/browser-matrix), displaying for which browsers your build passed/failed. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/other/browser-matrix # Browser Matrix With our Browser Matrix image you can quickly see the status for your latest test build for the various browsers/platforms. You can simply add markdown code or HTML code to your project website to show the status of your most recent test build. ## Builds In order to show a browser matrix image, you need to make sure that you're running your tests in a group, called a **build**. For example: you have a project that has a bunch of tests. After every commit you want to make sure the tests still pass. The tests you want to run are grouped in a **build**. To make sure your tests are grouped in the same build, use Selenium's Desired Capabilities. Simply pass `"build" : "build-1XXX"` in your Desired Capabilities: ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("platformName", "WIN11"); caps.setCapability("browserName", "Chrome"); caps.setCapability("build", "build-1XXX"); // for example ``` In our [member area](https://testingbot.com/members/builds) you will then see a new build, `build-1XXX`, which will contain all the tests that ran with the same build identifier. ## Marking tests passed/failed To mark tests as passed or failed, your test runner should be configured to send the success state of each test to TestingBot via the [TestingBot API](https://testingbot.com/support/api/tests#updatetest). The browser matrix image will then show the success state for every browser/platform in your build. ## Browser Matrix Below is an example of how the browser matrix image looks like, and information on how to embed this image on your website. ![TestingBot Build Matrix](https://testingbot.com/assets/support/buildmatrix-8f49f44a625e3b4118d16079fd18239f5ed45b30ed0ba283b9e33ec980ecbc4f.png) **Markdown:** ```markdown [![TestingBot Test Status](https://testingbot.com/buildmatrix/YOUR_TESTINGBOT_KEY?auth={authentication})](https://testingbot.com/members/builds) ``` **HTML:** ```html TestingBot Build Matrix ``` ## Private Accounts To display the matrix image, you need to generate an **authentication token**. This token only unlocks the badge and build matrix images; it does not give access to your builds or tests. To generate the authentication token, you must concatenate your TestingBot key, TestingBot secret and optionally an identifier. Finally, take the MD5 hash of the concatenated string: [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) ```ruby Digest::MD5.hexdigest("#{client_key}:#{client_secret}") ``` ```java MessageDigest m=MessageDigest.getInstance("MD5"); String s = "API_KEY:API_SECRET"; m.update(s.getBytes(),0,s.length()); System.out.println("MD5: "+new BigInteger(1,m.digest()).toString(16)); ``` ```php echo md5("API_KEY:API_SECRET"); ``` ```python import hashlib print(hashlib.md5("API_KEY:API_SECRET".encode('utf-8')).hexdigest()) ``` ```javascript var crypto = require('crypto'); crypto.createHash('md5').update("API_KEY:API_SECRET").digest("hex"); ``` ## Open Source Accounts [Open Source](https://testingbot.com/open-source) accounts automatically have their tests and builds set to public. This means no authentication (auth parameter) is required to display matrix images or link to tests/builds. ## More - If you like to show browser matrix images of **multiple projects** , please consider creating a new user account per project. - We're also providing [status badges](https://testingbot.com/support/other/status-badges), displaying if your build passed/failed. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/tunnel # Secure tunnel for local and staging tests A free, open-source secure proxy that lets you run Selenium, Cypress, Playwright, Puppeteer and Appium tests against **localhost** , staging servers or any environment behind a firewall or VPN. No IP whitelisting required. - v5.0 - Open Source - Free for every plan - Java 17+ or standalone `java -jar testingbot-tunnel.jar TESTINGBOT_KEY TESTINGBOT_SECRET` bash [Download v5.0](https://testingbot.com#downloads) [5-minute quickstart](https://testingbot.com/support/tunnel/quickstart) [View source on GitHub](https://github.com/testingbot/Testingbot-Tunnel) - **Released:** 2026-10-02 - **License:** MIT - **SHA-256 (zip):** `e0306bc1bf495f5836a9939b18b7c9ba768b28ce49b9de969e3b140423ad0090` - **Maven:** `com.testingbot:TestingBotTunnel:5.0` - **Docker:** `docker pull testingbot/tunnel:5.0` - **Source:** [`github.com/testingbot/Testingbot-Tunnel`](https://github.com/testingbot/Testingbot-Tunnel) ## What is TestingBot Tunnel? A Java application that proxies HTTP, HTTPS and WebSocket traffic between your network and the TestingBot browser and device grid. _TestingBot Tunnel architecture: your client running Selenium, Playwright or Puppeteer connects through an SSH-encrypted tunnel to the TestingBot grid of browsers and devices, with optional access to your staging or internal network_ When the tunnel is running, every request your test makes to `localhost:4445/wd/hub` is forwarded over an SSH-encrypted channel to a dedicated VM in the TestingBot cloud. That VM then routes browser traffic back through the tunnel to your local server, so the cloud browser can reach pages on your laptop, your CI runner or your private staging environment without any firewall or IP-whitelisting changes. At a glance - **Encryption:** SSH - **Protocols:** HTTP/1, /2, /3, WS, SSE - **Outbound ports:** 443 + 22 - **Architecture:** Java 17, standalone builds ## Quickstart Most teams have the tunnel running in under 5 minutes. The steps below use the JAR and assume Java 17+ is installed. No Java? Use a [self-contained build](https://testingbot.com#downloads). For Docker, Maven and the NodeJS launcher, see the [installation guide](https://testingbot.com/support/tunnel/installation). 01 Download the tunnel JAR One file, no daemon, no service to install. The JAR runs everywhere Java 17+ runs. ```bash curl -O https://testingbot.com/downloads/testingbot-tunnel.zip unzip testingbot-tunnel.zip ``` 02 Start the tunnel with your credentials Wait for the message `You may start your tests`. The tunnel provisions a dedicated VM in the TestingBot cloud and opens an SSH-encrypted channel. ```bash java -jar testingbot-tunnel.jar TESTINGBOT_KEY TESTINGBOT_SECRET ``` 03 Point your tests at the local tunnel That is the only change required. All Selenium, Appium, Playwright and Cypress capabilities work the same. ```diff - https://hub.testingbot.com/wd/hub + http://localhost:4445/wd/hub ``` Need more detail? Read the [extended quickstart guide](https://testingbot.com/support/tunnel/quickstart) with language-specific examples. ## Downloads Version 5.0, released 2026-10-02. The JAR needs Java 17 or newer. The self-contained builds bundle a trimmed Java runtime, so nothing has to be installed. Upgrading from 4.x? Read [what changes in 5.0](https://testingbot.com/support/tunnel/upgrading) first. | Platform | Download | Size | SHA-256 | | --- | --- | --- | --- | | **Any OS with Java 17+** | [`testingbot-tunnel.zip`](https://testingbot.com/downloads/testingbot-tunnel.zip) | 7.2 MB | `e0306bc1bf495f5836a9939b18b7c9ba768b28ce49b9de969e3b140423ad0090` | | **Linux x64** | [`testingbot-tunnel-linux-x64.tar.gz`](https://github.com/testingbot/Testingbot-Tunnel/releases/download/v5.0/testingbot-tunnel-linux-x64.tar.gz) | 31.1 MB | `88f4edcd4b88f174bdc89074ef2f24176dc4dfcff625f12fea0ea84748f47030` | | **Linux arm64** | [`testingbot-tunnel-linux-arm64.tar.gz`](https://github.com/testingbot/Testingbot-Tunnel/releases/download/v5.0/testingbot-tunnel-linux-arm64.tar.gz) | 30.8 MB | `a8a5e591799e5acbbd706a84f05259e0a79716a1bdae17d725da1a8fbc0cefb1` | | **macOS Apple silicon** | [`testingbot-tunnel-macos-arm64.tar.gz`](https://github.com/testingbot/Testingbot-Tunnel/releases/download/v5.0/testingbot-tunnel-macos-arm64.tar.gz) | 27.3 MB | `ae796c03b6ba8e332a9cc36035f4354afa456da4a35688d96e5986b58cbb93cf` | | **macOS Intel** | [`testingbot-tunnel-macos-x64.tar.gz`](https://github.com/testingbot/Testingbot-Tunnel/releases/download/v5.0/testingbot-tunnel-macos-x64.tar.gz) | 28.1 MB | `193720f150388b145cce8ddcc18ffb6b21a27b0d8a12d5f8e6f9b30f53f0fb72` | | **Windows x64** | [`testingbot-tunnel-windows-x64.zip`](https://github.com/testingbot/Testingbot-Tunnel/releases/download/v5.0/testingbot-tunnel-windows-x64.zip) | 28.9 MB | `7c6f1150e0732e4e20e4fcb63d75784f33c1f69aeb99366db6c584b4a190b67a` | Self-contained builds unpack to a folder with a `bin/testingbot-tunnel` launcher (`bin\testingbot-tunnel.cmd` on Windows) that takes the same options as the JAR. The macOS builds are signed and notarized. ```bash tar -xzf testingbot-tunnel-macos-arm64.tar.gz ./testingbot-tunnel-macos-arm64/bin/testingbot-tunnel --version ``` Also available as - Docker: `testingbot/tunnel:5.0` - Maven Central: `com.testingbot:TestingBotTunnel:5.0` - Every file with its `.sha256` on the [GitHub release](https://github.com/testingbot/Testingbot-Tunnel/releases/tag/v5.0) ## Key features Fast Static caching and compression keep round-trip latency low. [See monitoring](https://testingbot.com/support/tunnel/monitoring). Secure SSH-encrypted channel, dedicated ephemeral VM, automatic teardown. [Security model](https://testingbot.com/support/tunnel/security). Robust HTTP/1, /2, /3, WebSockets and SSE. Built for parallel workloads. [Troubleshoot](https://testingbot.com/support/tunnel/troubleshooting). Simple Single JAR, standalone binary or Docker image. Ready in under a minute. [Installation](https://testingbot.com/support/tunnel/installation). ## Documentation Focused guides covering everything from your first test to production monitoring. [Quickstart](https://testingbot.com/support/tunnel/quickstart) Run your first test through the tunnel in 5 minutes. [Installation](https://testingbot.com/support/tunnel/installation) JAR, standalone builds, Docker, Maven, NodeJS. [Upgrading to 5.0](https://testingbot.com/support/tunnel/upgrading) Java 17, loopback listeners, quieter logs. [Configuration](https://testingbot.com/support/tunnel/configuration) Config files and TESTINGBOT\_\* variables. [CLI reference](https://testingbot.com/support/tunnel/commandline) Every flag, every environment variable. [Examples](https://testingbot.com/support/tunnel/examples) Headers, identifiers, basic auth, custom DNS. [Multiple tunnels](https://testingbot.com/support/tunnel/multiple) Parallel test setups using tunnel identifiers. [Upstream proxy](https://testingbot.com/support/tunnel/upstream-proxy) HTTP and SOCKS5 proxies, PAC files. [Authentication](https://testingbot.com/support/tunnel/authentication) Basic auth and Kerberos for sites and proxies. [Traffic rules](https://testingbot.com/support/tunnel/traffic-rules) Allow-lists, fast-fail, localhost policy, DNS. [SSL bumping](https://testingbot.com/support/tunnel/ssl-bumping) Certificates, --nobump and corporate CAs. [Header rewriting](https://testingbot.com/support/tunnel/headers) Add, replace or remove HTTP headers. [Security](https://testingbot.com/support/tunnel/security) SSH encryption, dedicated VMs, certificates. [Health checks](https://testingbot.com/support/tunnel/health-checks) /readyz, Docker and Kubernetes probes. [Logging](https://testingbot.com/support/tunnel/logging) HTTP logs, JSON output, request IDs. [Monitoring](https://testingbot.com/support/tunnel/monitoring) Prometheus metrics, Grafana dashboards. [Embedding (Java)](https://testingbot.com/support/tunnel/embedding) Start and stop the tunnel from your tests. [Troubleshooting](https://testingbot.com/support/tunnel/troubleshooting) Ports, SSL bumping, slow speeds, disconnects. [FAQ](https://testingbot.com/support/tunnel/faq) Common questions about ports, devices, WS. [Changelog](https://testingbot.com/support/tunnel/changelog) Release notes and version history. [Source on GitHub](https://github.com/testingbot/Testingbot-Tunnel) Open-source project, MIT licensed. ## System requirements - Java 17+, or none with a standalone build - RAM 4 GB minimum - CPU 2 cores - OS Windows / macOS / Linux / BSD Outbound only The tunnel needs outbound port **443** (HTTPS) and **22** (SSH) to `*.testingbot.com`. No inbound ports required. For heavy parallel testing, scale CPU and RAM with concurrent sessions, or run [multiple tunnels](https://testingbot.com/support/tunnel/multiple). ## Graceful shutdown Stop the tunnel cleanly so the cloud can deregister your VM and free your tunnel slot. Option 1 Press `Ctrl+C` In the terminal that runs the tunnel. Option 2 Delete the PID file `~/.testingbot-tunnel.pid` Option 3 Send a signal `kill -SIGUSR1 ` ## Stopping a tunnel Stop a running tunnel from your terminal, the dashboard or the API, for example when a CI job exited without cleaning up. CLI Press `Ctrl+C` Or use any of the [graceful shutdown](https://testingbot.com#shutdown) options above. Dashboard Tunnels page, **Stop** button Open [/members/tunnels](https://testingbot.com/members/tunnels) and click Stop on the tunnel's row. API `DELETE /v1/tunnel/{id}` Get the id from `GET /v1/tunnel/list`. See the [API reference](https://testingbot.com/support/api#apitunneldelete). Who can stop a tunnel On a team, every member can see all of the team's tunnels. A tunnel can be stopped by: - the person who started it (the tunnel's **Owner** column), - the team **owner** , - any team member with the **Admin** role. The same rule applies to the dashboard and the API. Other members see the tunnel but no Stop button. ## Tunnel clients Launch and stop the tunnel programmatically from your test runner or CI pipeline. Two official launchers handle the tunnel lifecycle so you do not have to manage the JAR yourself: - **NodeJS:** [testingbot-tunnel-launcher](https://github.com/testingbot/testingbot-tunnel-launcher) — start and stop the tunnel from any npm-based test runner. - **GitHub Actions:** [testingbot-tunnel-action](https://github.com/testingbot/testingbot-tunnel-action) — drop-in workflow step that boots the tunnel before your tests and tears it down after. See the [GitHub Actions guide](https://testingbot.com/support/integrations/ci-cd/github-actions). The tunnel also ships as a Docker image and as a Maven dependency. See the [installation guide](https://testingbot.com/support/tunnel/installation) for all options. [![TestingBot Tunnel architecture diagram](https://testingbot.com/assets/tunnel-diagram-small-af64aa8c1e0d3558a0cc730b3b7cc827264e9af90173acb7627cff04054e1d89.jpg)](https://testingbot.com/assets/tunnel-diagram.png "How the TestingBot Tunnel works") ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/tunnel/quickstart # Your first tunneled test in 5 minutes This guide gets a Selenium, Cypress, Playwright, Puppeteer or Appium test running against your local development server. If you have not already, [create a free TestingBot account](https://testingbot.com/users/sign_up) before you start. _TestingBot Tunnel architecture: your client running Selenium, Playwright or Puppeteer connects through an SSH-encrypted tunnel to the TestingBot grid of browsers and devices, with optional access to your staging or internal network_ ## Prerequisites - JAVA 17 or higher. Check with `java -version`. No Java? Use a [standalone build](https://testingbot.com/support/tunnel/installation#standalone) or [Docker](https://testingbot.com#docker). - ACCOUNT A TestingBot account. The free trial is enough. - CREDENTIALS Your TestingBot key and secret from [account settings](https://testingbot.com/members/user/edit). - LOCAL SERVER Anything that responds on a local port, e.g. `localhost:3000`. ## Download the tunnel Download the latest release and unzip it. Place `testingbot-tunnel.jar` anywhere on your `PATH`, or run it from the folder you unzipped it in. ```bash curl -O https://testingbot.com/downloads/testingbot-tunnel.zip unzip testingbot-tunnel.zip ``` Prefer Docker or Maven? See the [installation guide](https://testingbot.com/support/tunnel/installation) for every install method. ## Start the tunnel Pass your TestingBot key and secret. You can also export them as environment variables so they never appear in your shell history. ```bash java -jar testingbot-tunnel.jar TESTINGBOT_KEY TESTINGBOT_SECRET ``` You will see log output as the tunnel provisions a dedicated VM in the TestingBot cloud and opens an SSH-encrypted channel. After 30 to 60 seconds, you will see: ```log You may start your tests ``` The tunnel is now listening on `localhost:4445` for your Selenium and Appium clients. ## Run your first test Change the hub URL in your test runner from the public TestingBot hub to your local tunnel endpoint. That is the only change required. ```diff - https://hub.testingbot.com/wd/hub + http://localhost:4445/wd/hub ``` Run your tests as you normally would, and the cloud browser will reach your `localhost` via the tunnel. ## Language examples The same first test in five common languages. Each one points the WebDriver at the local tunnel hub. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [C#](https://testingbot.com#) ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("browserName", "chrome"); caps.setCapability("platformName", "WIN11"); WebDriver driver = new RemoteWebDriver( new URL("http://KEY:SECRET@localhost:4445/wd/hub"), caps); driver.get("http://localhost:3000"); System.out.println(driver.getTitle()); driver.quit(); ``` ```python from selenium import webdriver options = webdriver.ChromeOptions() options.set_capability("platformName", "WIN11") driver = webdriver.Remote( command_executor="http://KEY:SECRET@localhost:4445/wd/hub", options=options, ) driver.get("http://localhost:3000") print(driver.title) driver.quit() ``` ```javascript const { remote } = require("webdriverio"); const browser = await remote({ hostname: "localhost", port: 4445, path: "/wd/hub", user: "KEY", key: "SECRET", capabilities: { browserName: "chrome", platformName: "WIN11" }, }); await browser.url("http://localhost:3000"); console.log(await browser.getTitle()); await browser.deleteSession(); ``` ```ruby require "selenium-webdriver" caps = Selenium::WebDriver::Remote::Capabilities.new caps["browserName"] = "chrome" caps["platformName"] = "WIN11" driver = Selenium::WebDriver.for(:remote, url: "http://KEY:SECRET@localhost:4445/wd/hub", capabilities: caps) driver.navigate.to "http://localhost:3000" puts driver.title driver.quit ``` ```csharp var options = new ChromeOptions(); options.AddAdditionalCapability("platformName", "WIN11", true); IWebDriver driver = new RemoteWebDriver( new Uri("http://KEY:SECRET@localhost:4445/wd/hub"), options); driver.Navigate().GoToUrl("http://localhost:3000"); Console.WriteLine(driver.Title); driver.Quit(); ``` ## Run with Docker Prefer not to install Java locally? The official `testingbot/tunnel` image on Docker Hub bundles everything you need. It supports both `amd64` and `arm64` and is the easiest way to run the tunnel from CI or a containerised dev environment. 1. **Pull the image** from Docker Hub. 2. **Start the tunnel** , passing your TestingBot key and secret as environment variables. 3. **Point your tests** at `http://localhost:4445/wd/hub`, exactly the same as the JAR version. ```bash docker pull testingbot/tunnel:5.0 docker run --rm -it \ --network host \ -e TESTINGBOT_BIND_ADDRESS=127.0.0.1 \ -e TESTINGBOT_KEY=YOUR_KEY \ -e TESTINGBOT_SECRET=YOUR_SECRET \ testingbot/tunnel:5.0 ``` `--network host` only works fully on Linux. On macOS and Windows, publish the ports explicitly, on the loopback address, so your tests can reach the tunnel on `localhost:4445` without exposing it to your network: ```bash docker run --rm -it \ -p 127.0.0.1:4445:4445 -p 127.0.0.1:8087:8087 -p 127.0.0.1:8003:8003 \ -e TESTINGBOT_KEY=$TESTINGBOT_KEY \ -e TESTINGBOT_SECRET=$TESTINGBOT_SECRET \ testingbot/tunnel:5.0 ``` Once the container prints `You may start your tests`, the tunnel is ready. Stop it gracefully with `Ctrl+C`. For Compose, Maven and the NodeJS launcher, see the [installation guide](https://testingbot.com/support/tunnel/installation). ## Next steps [Multiple tunnels](https://testingbot.com/support/tunnel/multiple) Add identifiers to run more than one tunnel at a time. [Upstream proxy](https://testingbot.com/support/tunnel/upstream-proxy) Forward through a corporate or GeoIP proxy. [Monitoring](https://testingbot.com/support/tunnel/monitoring) Configure Prometheus and Grafana. [Security model](https://testingbot.com/support/tunnel/security) Review SSH encryption and credential handling. [Troubleshooting](https://testingbot.com/support/tunnel/troubleshooting) If something goes wrong, this is where to look. [FAQ](https://testingbot.com/support/tunnel/faq) Ports, devices, WebSockets and more. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/tunnel/installation # Install the tunnel Pick the flavour that fits your environment best. The single-file JAR works everywhere with Java 17+. The standalone builds for Linux, macOS and Windows need no Java at all. Docker is the easiest way to run the tunnel in CI. The Maven artifact and NodeJS launcher let you embed the tunnel in your existing build tooling. ## Before you install - JAVA 17 or higher for the JAR. Verify with `java -version`. Not needed for the [standalone builds](https://testingbot.com#standalone) or Docker. - NETWORK Outbound access to `*.testingbot.com` on ports 443 and 22. - CREDENTIALS A TestingBot key and secret from [account settings](https://testingbot.com/members/user/edit). ## JAR One file, no daemon, no service to install. Runs on any OS with Java 17 or newer. ```bash curl -O https://testingbot.com/downloads/testingbot-tunnel.zip unzip testingbot-tunnel.zip java -jar testingbot-tunnel.jar TESTINGBOT_KEY TESTINGBOT_SECRET ``` Verify the download checksum before running: ```bash shasum -a 256 testingbot-tunnel.jar # expected: 171fb4012ed2375313c5b6a63c305fa1c6e33e0371c3eb4c7304ecee935492f8 ``` ## Standalone builds Self-contained archives for Linux, macOS and Windows that bundle a trimmed Java runtime. Unpack and run, nothing else to install. Each archive has a `.sha256` file next to it on the [GitHub release](https://github.com/testingbot/Testingbot-Tunnel/releases/tag/v5.0). Linux [x64](https://github.com/testingbot/Testingbot-Tunnel/releases/download/v5.0/testingbot-tunnel-linux-x64.tar.gz) · [arm64](https://github.com/testingbot/Testingbot-Tunnel/releases/download/v5.0/testingbot-tunnel-linux-arm64.tar.gz) macOS [Apple silicon](https://github.com/testingbot/Testingbot-Tunnel/releases/download/v5.0/testingbot-tunnel-macos-arm64.tar.gz) · [Intel](https://github.com/testingbot/Testingbot-Tunnel/releases/download/v5.0/testingbot-tunnel-macos-x64.tar.gz). Signed and notarized. Windows [x64](https://github.com/testingbot/Testingbot-Tunnel/releases/download/v5.0/testingbot-tunnel-windows-x64.zip) Linux and macOS: ```bash curl -LO https://github.com/testingbot/Testingbot-Tunnel/releases/download/v5.0/testingbot-tunnel-linux-x64.tar.gz curl -LO https://github.com/testingbot/Testingbot-Tunnel/releases/download/v5.0/testingbot-tunnel-linux-x64.tar.gz.sha256 sha256sum -c testingbot-tunnel-linux-x64.tar.gz.sha256 # macOS: shasum -a 256 -c tar -xzf testingbot-tunnel-linux-x64.tar.gz ./testingbot-tunnel-linux-x64/bin/testingbot-tunnel TESTINGBOT_KEY TESTINGBOT_SECRET ``` Windows (PowerShell): ```powershell Invoke-WebRequest -OutFile testingbot-tunnel-windows-x64.zip https://github.com/testingbot/Testingbot-Tunnel/releases/download/v5.0/testingbot-tunnel-windows-x64.zip Expand-Archive testingbot-tunnel-windows-x64.zip -DestinationPath . .\testingbot-tunnel-windows-x64\bin\testingbot-tunnel.cmd TESTINGBOT_KEY TESTINGBOT_SECRET ``` The launcher takes exactly the same options as the JAR, so every example in these docs works by replacing `java -jar testingbot-tunnel.jar` with `bin/testingbot-tunnel`. ## Docker The official image is published as `testingbot/tunnel` on Docker Hub and supports both amd64 and arm64. ```bash docker pull testingbot/tunnel:5.0 docker run --rm -it \ --network host \ -e TESTINGBOT_BIND_ADDRESS=127.0.0.1 \ -e TESTINGBOT_KEY=your_key \ -e TESTINGBOT_SECRET=your_secret \ testingbot/tunnel:5.0 ``` The image listens on every interface inside the container, so that published ports work. With `--network host` the container shares your machine's interfaces, which is why the command above sets `TESTINGBOT_BIND_ADDRESS=127.0.0.1`: the Selenium relay carries your credentials and should not be reachable from your network. `--network host` only works fully on Linux. On macOS and Windows, publish the ports explicitly, on the loopback address: ```bash docker run --rm -it \ -p 127.0.0.1:4445:4445 -p 127.0.0.1:8087:8087 -p 127.0.0.1:8003:8003 \ -e TESTINGBOT_KEY=$TESTINGBOT_KEY \ -e TESTINGBOT_SECRET=$TESTINGBOT_SECRET \ testingbot/tunnel:5.0 ``` Almost every tunnel option can be passed as a `TESTINGBOT_*` environment variable, see [configuration](https://testingbot.com/support/tunnel/configuration#env). The image has a built-in health check, see [health checks](https://testingbot.com/support/tunnel/health-checks#docker). ## Maven Add the tunnel as a dependency to your `pom.xml` and start it from a JUnit or TestNG fixture. See [embedding](https://testingbot.com/support/tunnel/embedding) for the API. ```xml com.testingbot TestingBotTunnel 5.0 ``` ## NodeJS launcher For npm-based projects, use the `testingbot-tunnel-launcher` package to start and stop the tunnel programmatically. The launcher handles downloading the right JAR for your platform. ```bash npm install --save-dev testingbot-tunnel-launcher ``` ```javascript const testingbotTunnel = require("testingbot-tunnel-launcher"); testingbotTunnel({ apiKey: process.env.TESTINGBOT_KEY, apiSecret: process.env.TESTINGBOT_SECRET, }, (err, tunnel) => { if (err) throw err; // your tests here tunnel.close(() => console.log("tunnel closed")); }); ``` Source on GitHub: [testingbot-tunnel-launcher](https://github.com/testingbot/testingbot-tunnel-launcher). ## GitHub Action If your tests run in GitHub Actions, the official `testingbot/testingbot-tunnel-action` handles the entire tunnel lifecycle for you. It pulls the latest Docker image, starts the tunnel before your test step, waits for the ready signal, and tears it down when the workflow ends. Add it as a step in your workflow, before the step that runs your tests: ```yaml jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: testingbot/testingbot-tunnel-action@v1 with: key: ${{ secrets.TB_KEY }} secret: ${{ secrets.TB_SECRET }} tunnelIdentifier: github-action-tunnel - name: Run tests run: npm test env: TB_KEY: ${{ secrets.TB_KEY }} TB_SECRET: ${{ secrets.TB_SECRET }} ``` The action supports every tunnel flag as a workflow input: `tunnelIdentifier`, `noBump`, `noCache`, `proxy`, `extraHeaders`, `fastFailRegexps` and more. It also uploads the tunnel log as a workflow artifact by default for easy debugging. Full reference and example workflows: [GitHub Actions integration guide](https://testingbot.com/support/integrations/ci-cd/github-actions). Source: [testingbot-tunnel-action](https://github.com/testingbot/testingbot-tunnel-action). ## Sanity check with --doctor Once the tunnel is installed, run the built-in health check. It verifies that the required local ports are free, that outbound connectivity to TestingBot works, and that your credentials are valid. ```bash java -jar testingbot-tunnel.jar --doctor ``` Pass the same options you normally start the tunnel with: `--doctor` checks the configuration it is given, including upstream proxies, Kerberos and any `--cacert-file`. If all checks pass, continue with the [quickstart](https://testingbot.com/support/tunnel/quickstart). If anything fails, the [troubleshooting guide](https://testingbot.com/support/tunnel/troubleshooting) covers the most common causes. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/tunnel/examples # Configuration recipes Practical recipes for the most common TestingBot Tunnel configurations. Start with the basic setup, then add identifiers, headers, proxies and basic auth as your tests need them. ## Getting started Download the tunnel and start it from the command line. ```bash java -jar testingbot-tunnel.jar key secret ``` Then update your test configuration to route traffic through the tunnel by changing the hub URL: ```diff - urlhub = "https://key:secret@hub.testingbot.com/wd/hub" + urlhub = "http://key:secret@localhost:4445/wd/hub" ``` By connecting to `localhost:4445`, all test commands are routed through the tunnel to TestingBot. ## Tunnel identifiers Run multiple tunnels simultaneously and direct specific tests to a particular tunnel. ```bash java -jar testingbot-tunnel.jar key secret -i myTunnel ``` Then specify the identifier in your test's desired capabilities: ```json { "browserName": "firefox", "browserVersion": "latest", "platformName": "WIN11", "tunnel-identifier": "myTunnel" } ``` See [running multiple tunnels](https://testingbot.com/support/tunnel/multiple) for parallel and high-availability setups. ## Custom headers Inject custom HTTP headers into every request the tunnel makes. Useful for authentication tokens, feature flags, A/B test cookies and debugging headers. ```bash java -jar testingbot-tunnel.jar key secret \ --extra-headers '{"MyOwnHeader": "HeaderValue"}' ``` ## Upstream proxy Route tunnel traffic through an upstream proxy server. Common use cases: - Corporate networks that require all outbound traffic to pass through a proxy. - Geo-location testing using HTTP proxies in different regions. Use `-Y` for the proxy address and `-z` for credentials: ```bash java -jar testingbot-tunnel.jar key secret -Y myproxy:9000 -z username:password ``` For PAC files and more advanced setups, see the [upstream proxy guide](https://testingbot.com/support/tunnel/upstream-proxy). ## Shared tunnels Tunnels are private by default. Enable the `--shared` flag to make a tunnel available to other team members on the same TestingBot account. ```bash java -jar testingbot-tunnel.jar key secret --shared ``` Team members can then run their tests against the shared tunnel without starting their own. Combine with `--tunnel-identifier` to give the shared tunnel a memorable name. ## Basic authentication Automatically supply credentials for websites that require HTTP Basic Authentication. The tunnel injects these credentials for matching hosts, so your tests do not have to handle authentication dialogs. Use the `-a` flag with the format `host:port:username:password`. For standard HTTP sites use port 80: ```bash java -jar testingbot-tunnel.jar key secret -a mywebsite.com:80:myusername:mypassword ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/tunnel/multiple # More than one tunnel, in parallel Your account supports up to **2** simultaneous tunnels. They can be tunnels you start yourself or shared tunnels from team members on the same account. ## Why use multiple tunnels? Parallel Higher throughput Spread test traffic across tunnels for higher throughput. HA High availability Run tunnels on different machines for redundancy. Isolate Environment isolation Dedicate one tunnel per environment (staging, QA, dev). Team Team workflows Let team members run tunnels for their own tests without conflicts. ## Setting up tunnel identifiers Use the `--tunnel-identifier` flag to give each tunnel a unique name. ```bash java -jar testingbot-tunnel.jar key secret --tunnel-identifier myTunnel ``` Then specify the identifier in the desired capabilities of each test that should be routed through that tunnel: ```json { "browserName": "firefox", "browserVersion": "latest", "platformName": "WIN11", "tunnel-identifier": "myTunnel" } ``` ## Parallel test isolation When two test suites both rely on the tunnel (for example a staging suite and a QA suite), give each suite its own tunnel and identifier. This prevents one suite from saturating the other's bandwidth and makes monitoring per-suite traffic straightforward via the [metrics endpoint](https://testingbot.com/support/tunnel/monitoring). ## High availability Run identically-configured tunnels on two or more machines using the same identifier. The TestingBot grid load-balances test traffic across all available tunnels with that identifier and routes around any tunnel that becomes unhealthy. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/tunnel/upstream-proxy # Route through an upstream proxy On many networks the only route out is a proxy. The tunnel can send its own connection to TestingBot, and the traffic it carries for your tests, through an upstream HTTP or SOCKS5 proxy. It can also use a different proxy per destination. Common uses are corporate proxies and GeoIP proxies for region-specific testing. - HTTP - SOCKS5 - PAC - Kerberos ## Basic configuration Pass the proxy with `--proxy`, and credentials with `--proxy-userpwd`. ```bash java -jar testingbot-tunnel.jar --proxy --proxy-userpwd : ``` | Form | Proxy type | | --- | --- | | host:port | HTTP proxy | | http://host:port | HTTP proxy | | socks5://host:port | SOCKS5 proxy, with optional username and password | Unless you set [`--proxy-testingbot`](https://testingbot.com#split), the proxy carries everything: the browser's requests to your sites, HTTPS `CONNECT` tunnels, WebSocket upgrades, and the tunnel's own connection to TestingBot. ## Corporate proxies Forward both the control channel to `*.testingbot.com` and the test traffic through your authenticated proxy. Export `TESTINGBOT_PROXY_USERPWD` instead of passing `-z` on the command line so the credentials never appear in the process list. ```bash export TESTINGBOT_PROXY_USERPWD=alice:hunter2 java -jar testingbot-tunnel.jar --proxy corp-proxy.example.com:8080 ``` Proxy needs Kerberos instead of a password? Add `--proxy-auth-scheme negotiate`. See [Kerberos proxies](https://testingbot.com/support/tunnel/authentication#kerberos-proxy). ## A separate proxy for TestingBot Where egress is filtered by destination, the proxy allowed out to the internet is often not the one that reaches your internal test targets. `--proxy-testingbot` gives TestingBot's own traffic, the API and the tunnel's SSH connection, a different proxy from the one test traffic uses. ```bash java -jar testingbot-tunnel.jar \ --proxy 10.0.0.9:8080 \ --proxy-testingbot corp-egress:3128 --proxy-testingbot-userpwd user:password ``` Credentials are deliberately not shared: `--proxy-userpwd` belongs to one proxy operator and is not sent to another. Give each proxy its own. ## Routing with a PAC file `--pac-local` points the tunnel at a proxy auto-config file and lets it decide, per destination, whether to go direct or through a proxy. This is how your browser already handles a corporate network, applied to the tunnel. ```bash java -jar testingbot-tunnel.jar --pac-local /etc/corp.pac java -jar testingbot-tunnel.jar --pac-local https://corp.example/proxy.pac ``` Check what a file decides for a URL before relying on it: ```bash java -jar testingbot-tunnel.jar --pac-local corp.pac --pac-test https://internal.corp/page ``` - **Use `https://` or a local file.** A PAC file decides where traffic goes. Fetched over plain `http://`, it is whatever the network says it is, so a plain `http://` URL is refused unless you pin its contents with `--pac-local-sha256`. Redirects are refused rather than followed. - **A purpose-built interpreter, no JavaScript engine.** It supports what PAC files use: functions, variables, conditionals, loops, the usual operators, and the standard helpers (`isPlainHostName`, `dnsDomainIs`, `shExpMatch`, `isInNet`, `dnsResolve`, `myIpAddress` and the date and time predicates). - **Unsupported syntax is reported with its line number** , rather than guessed at. Object literals, regular expressions and `new` are not supported. A misread PAC file would silently send traffic to the wrong place. - **No credentials for PAC-chosen proxies.** A proxy named only by the PAC file does not receive `--proxy-userpwd`. The tunnel logs that it withheld them. ```bash java -jar testingbot-tunnel.jar --pac-local http://wpad.corp/proxy.pac \ --pac-local-sha256 3b1f...64-hex-characters ``` ## PAC for the remote browser `--pac` is a different option: it tells the **cloud browser** which PAC URL to use. It does not change where the tunnel itself sends traffic. ```bash java -jar testingbot-tunnel.jar --pac https://corp.example.com/proxy.pac ``` | Option | Evaluated by | Decides | | --- | --- | --- | | --pac-local | The tunnel, on your machine | Where the tunnel sends its own outgoing traffic | | --pac | The remote browser | Which proxy the cloud browser uses | ## TLS-intercepting proxies If the proxy intercepts TLS and re-signs it with an internal certificate authority, Java will not trust it and the tunnel cannot register itself. Point it at the authority. ```bash java -jar testingbot-tunnel.jar --proxy proxy.corp:8080 --cacert-file /etc/ssl/corp-ca.pem ``` The certificate is added to the platform's trust store rather than replacing it. `--doctor` prints the subjects it loaded. See [SSL bumping and certificates](https://testingbot.com/support/tunnel/ssl-bumping#cacert). ## WebSockets through a proxy `--ws-proxy-mode` controls how a `ws://` upgrade crosses `--proxy`. | Mode | Behaviour | | --- | --- | | connect | **Default.** Ask the proxy to open a tunnel with `CONNECT` and upgrade inside it. This is what RFC 6455 specifies and what browsers do. | | get | Send the upgrade to the proxy as an ordinary request. For proxies that forward `Upgrade` but only allow `CONNECT` to port 443. | `wss://` is always a `CONNECT`, so this option does not affect it. ## Things to know - Run `--doctor` with your proxy options. It tests the route out and, for Kerberos, every step of the Negotiate chain. - A failing proxy shows up in the cloud browser with an `X-TestingBot-Error` of `upstream-proxy-unreachable`, `upstream-proxy-auth-failed` or `upstream-proxy-refused`. See [reading errors](https://testingbot.com/support/tunnel/traffic-rules#errors). - WebSocket and SSE traffic requires `--nobump --nocache` in addition to the proxy flags. - See the [CLI reference](https://testingbot.com/support/tunnel/commandline#flags-proxy) for every proxy-related flag. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/tunnel/security # Built for production traffic Every tunnel session runs inside its own ephemeral virtual machine, all traffic is encrypted in transit, and credentials never have to appear on the command line. - SSH encrypted - Dedicated VM - Ephemeral - MIT licensed ## Encrypted channel The tunnel opens a single outbound SSH connection from your machine to a TestingBot edge endpoint. All test traffic (HTTP, HTTPS, WebSocket, SSE) is multiplexed over that SSH connection. Inbound ports on your network are not required. The tunnel only initiates outbound connections to ports **443** (HTTPS) and **22** (SSH) on `*.testingbot.com`. ## Pinning the tunnel server's host key The tunnel's control connection is SSH, and your account secret is its password. Whatever answers on the tunnel port receives it, so the tunnel can verify the server's SSH host key before it sends anything. ```bash java -jar testingbot-tunnel.jar --ssh-host-key SHA256:xxxx... --ssh-host-key-policy require ``` - `--ssh-host-key` pins the SHA-256 fingerprint the server must present, in the form `ssh-keygen -lf -E sha256` prints. Repeat it, or comma-separate several, where the server may present more than one key. MD5 fingerprints are refused. - Where the TestingBot API has a fingerprint for your tunnel server, it sends it over its authenticated HTTPS connection and the tunnel checks it. A pin you give outranks it. - With neither, the tunnel connects anyway and logs a warning on every connect, which is what 4.x tunnels did. `--ssh-host-key-policy require` refuses to connect instead. - `--doctor` reports which of these cases applies before anything connects. ## Local listeners bind loopback The Selenium relay, local proxy, metrics port and `--web` server do not authenticate. Since 5.0 they listen on `127.0.0.1` only. That matters because the Selenium relay attaches your TestingBot key and secret to everything it forwards, and the local proxy will connect anywhere your machine can. If tests run on another machine, set `--bind-address 0.0.0.0` and restrict the ports with a firewall. See [upgrading](https://testingbot.com/support/tunnel/upgrading#bind). ## Limiting what the tunnel can reach A tunnel can reach whatever the machine running it can reach. On a CI runner or jump host, narrow that to what your tests need. ```bash java -jar testingbot-tunnel.jar --allow-hosts 'staging.example.com,*.internal.example' --localhost-policy deny ``` Both are enforced on plain HTTP, `CONNECT` and WebSocket upgrades. The localhost check is applied to the address actually dialled, so a DNS answer pointing at `127.0.0.1` cannot get around it. See [traffic rules](https://testingbot.com/support/tunnel/traffic-rules). ## Dedicated VM lifecycle Every tunnel session provisions a dedicated virtual machine in the TestingBot cloud. The VM exists only for the lifetime of your tunnel. 01 Provision A fresh VM is allocated when your tunnel starts. 02 Use Only your account (or your team if `--shared`) can route traffic. 03 Teardown The VM is destroyed when the tunnel shuts down. No state persists. VMs are not reused between accounts. There is no cross-tenant traffic on the tunnel VM. ## Certificates and SSL bumping By default, the tunnel performs SSL interception (also called SSL bumping) for outbound HTTPS so that the cloud browser can validate certificates seamlessly. The TestingBot CA used for bumping is preinstalled on every cloud VM and device. For apps that pin certificates, you have two options: Trust the TB CA [Download the certificate](https://testingbot.com/downloads/testingbot-ssl-bumping.crt) and add it to your app's trust store. Disable bumping Start the tunnel with `--nobump` and HTTPS is forwarded transparently with no decryption. ## Credential handling Your TestingBot key and secret authenticate the tunnel against the TestingBot grid. Treat them like passwords. The tunnel accepts credentials from three sources, in order of precedence: 1. Positional arguments on the command line. 2. Environment variables: `TESTINGBOT_KEY` and `TESTINGBOT_SECRET`. 3. A `.testingbot` file in your `$HOME` directory. For CI pipelines and shared machines, prefer the environment-variable approach so credentials never appear in shell history, process lists or build logs. See the [CLI reference](https://testingbot.com/support/tunnel/commandline#env) for the full list. ## Credentials stay out of logs Tunnel logs end up in CI artifacts and support tickets, so credentials are redacted before anything is written. - Credential-bearing headers such as `Authorization`, `Proxy-Authorization` and cookies. - The `user:password@` part of URLs and credential-shaped query values. - Capabilities logged with `--log-http forwarder:body`, by key name. - Upstream proxy credentials are only ever sent to the proxy they were issued for, never to a proxy named by a PAC file. See [authentication](https://testingbot.com/support/tunnel/authentication#scoping). ## Metrics endpoint The tunnel exposes a Prometheus-compatible metrics endpoint on port 8003 by default. The endpoint is unauthenticated unless you set `--metrics-auth user:password` (or `TESTINGBOT_METRICS_AUTH`). Since 5.0 the endpoint binds `127.0.0.1` by default. If you set `--bind-address 0.0.0.0`, enable metrics auth and restrict the port with a firewall rule. The `/healthz` and `/readyz` endpoints are not behind metrics auth, and disclose only up or down. See the [monitoring guide](https://testingbot.com/support/tunnel/monitoring). ## Data residency Test traffic flowing through the tunnel is routed via TestingBot edge endpoints in the EU. No traffic is stored after the tunnel ends. For specific data-residency or compliance information, please see the [TestingBot trust center](https://trust.testingbot.com). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/tunnel/monitoring # Watch your tunnel from Grafana Production tunnels are easier to operate when you can see their health at a glance. TestingBot Tunnel ships with a built-in Prometheus-compatible metrics endpoint, so you can scrape it from any monitoring stack you already run. - Prometheus - Grafana ready - Port 8003 ## Enable the metrics endpoint The metrics endpoint is enabled by default on `http://localhost:8003`. Override the port with `--metrics-port`. ```bash java -jar testingbot-tunnel.jar --metrics-port 9100 ``` If the tunnel host is reachable from the network (for example a shared CI runner), protect the endpoint with HTTP Basic Auth: ```bash java -jar testingbot-tunnel.jar --metrics-auth ops:s3cret ``` Export `TESTINGBOT_METRICS_AUTH` instead of passing credentials on the command line. See the [security guide](https://testingbot.com/support/tunnel/security#metrics). Since 5.0 the metrics port binds `127.0.0.1`. For Prometheus on another host, start the tunnel with `--bind-address 0.0.0.0`. The same port also serves `/healthz` and `/readyz` for liveness and readiness checks, see [health checks](https://testingbot.com/support/tunnel/health-checks). ## Available metrics Tunnel-specific series are prefixed with `testingbot_`. The endpoint also exposes the standard JVM and process metrics from the Prometheus Java client. The full label set is always documented at the `/metrics` endpoint itself. ### Tunnel state | Metric | Type | Meaning | | --- | --- | --- | | testingbot\_tunnel\_up | gauge | `1` when the tunnel is connected, `0` while reconnecting or down. The single most important alerting signal. | | testingbot\_tunnel\_info | info | Static labels with the tunnel build (`version`, `id`, `name`). Use for dashboard headers and version filters. | | testingbot\_tunnel\_uptime\_seconds | counter | Seconds since this tunnel process started. Drops on restart, useful for detecting flapping. | | testingbot\_tunnel\_reconnects\_total | counter | Total tunnel reconnects since startup. Sustained increases indicate an unstable upstream link. | | testingbot\_active\_connections | gauge | Number of in-flight client connections through the tunnel right now. | | testingbot\_tunnel\_connect\_duration\_seconds | histogram | Time taken to establish the tunnel itself (cold-start latency). Buckets `_bucket`, `_sum`, `_count`. | ### HTTP traffic | Metric | Type | Meaning | | --- | --- | --- | | testingbot\_http\_requests\_total | counter | HTTP requests proxied since startup. Labels: `method`, `code`. Use `rate()` for throughput, filter on `code=~"5.."` for errors. | | testingbot\_http\_request\_duration\_seconds | histogram | End-to-end HTTP latency. Compute p50/p95/p99 with `histogram_quantile()`. | | testingbot\_https\_connect\_total | counter | HTTPS CONNECT sessions established. Labels: `code`. | | testingbot\_https\_connect\_duration\_seconds | histogram | CONNECT handshake latency, suitable for `histogram_quantile()`. | | testingbot\_https\_connect\_errors\_total | counter | CONNECT errors. Label: `reason` (TLS, target unreachable, timeout, ...). | | testingbot\_proxy\_bytes\_transferred\_total | counter | Total bytes proxied (both directions). Use `rate()` for throughput in B/s. | | testingbot\_errors\_total | counter | Generic proxy errors. Label: `name` (the error class). Useful for alerting on burst increases. | ## JVM and process metrics The endpoint also exposes the standard `jvm_*` and `process_*` series from the Prometheus Java client. These help you size the tunnel host, detect memory pressure and catch garbage-collection pauses. | Metric | Type | Meaning | | --- | --- | --- | | jvm\_memory\_bytes\_used | gauge | Bytes of heap and non-heap memory in use. Label: `area` (`heap` or `nonheap`). | | jvm\_memory\_bytes\_max | gauge | Maximum bytes available per memory area. Pair with `_used` to compute headroom. | | jvm\_memory\_pool\_bytes\_used | gauge | Per-pool memory usage (Eden, Survivor, Old Gen, Metaspace, ...). Label: `pool`. | | jvm\_gc\_collection\_seconds\_count | counter | Number of GC collections. Label: `gc` (collector name). | | jvm\_gc\_collection\_seconds\_sum | counter | Total time spent in GC. Use `rate()` to see GC pressure over time. | | jvm\_threads\_current | gauge | Live thread count. Watch for runaway growth. | | jvm\_threads\_daemon | gauge | Daemon thread count. | | jvm\_threads\_peak | gauge | Peak thread count since process start. | | jvm\_classes\_loaded | gauge | Currently loaded classes. | | jvm\_buffer\_pool\_used\_bytes | gauge | Direct and mapped NIO buffer usage. Label: `pool`. | | process\_cpu\_seconds\_total | counter | Total CPU time consumed by the process. Use `rate()` for CPU utilisation. | | process\_resident\_memory\_bytes | gauge | RSS memory of the tunnel process as reported by the OS. | | process\_open\_fds | gauge | Open file descriptors. Compare with `process_max_fds` to spot leaks. | | process\_start\_time\_seconds | gauge | Unix epoch when the process started. Useful for restart detection. | All names above are the canonical Prometheus Java client names. Open `http://localhost:8003/metrics` to see the full live list and their `# HELP` / `# TYPE` annotations. ## Prometheus scrape config Add a job to your `prometheus.yml` that targets the tunnel host. ```yaml scrape_configs: - job_name: testingbot_tunnel static_configs: - targets: ["tunnel-host.example.com:8003"] basic_auth: username: ops password_file: /etc/prometheus/testingbot_metrics_password ``` ## Grafana dashboard A ready-made Grafana dashboard ships with the tunnel source. Import it once and you have a full Overview / HTTP / Tunnel-health view in seconds. Download `testingbot-tunnel.json` from the [grafana-dashboard examples folder](https://github.com/testingbot/Testingbot-Tunnel/tree/master/examples/grafana-dashboard) on GitHub. In Grafana go to **Dashboards → New → Import** , paste the JSON or upload the file, pick your Prometheus data source, and click **Import**. [![TestingBot Tunnel Grafana dashboard with Overview and HTTP panels](https://testingbot.com/assets/support/tunnel/grafana-dashboard-bf173520c136608330e3e59c0444883d41815ea3bd031cf063f3147f3ef65bf2.png) ](https://testingbot.com/assets/support/tunnel/grafana-dashboard-bf173520c136608330e3e59c0444883d41815ea3bd031cf063f3147f3ef65bf2.png) _The bundled dashboard rendered against a live tunnel_ The bundled dashboard groups panels into two rows: - **Overview:** Tunnel status (UP/DOWN), build version, active connections, uptime and reconnect counter. - **HTTP:** request rate by status class, HTTPS CONNECT rate, p50/p95/p99 latency and response throughput. Want a turnkey setup? The repo also includes a [docker-compose example](https://github.com/testingbot/Testingbot-Tunnel/tree/master/examples/docker-compose-prometheus-grafana) that spins up Prometheus + Grafana already wired to scrape a local tunnel and load the dashboard automatically. Useful for local development or a single-machine CI runner. ```bash git clone https://github.com/testingbot/Testingbot-Tunnel.git cd Testingbot-Tunnel/examples/docker-compose-prometheus-grafana docker compose up -d # Grafana on http://localhost:3000 · Prometheus on http://localhost:9090 ``` If you prefer to build the dashboard yourself, here are the most useful PromQL queries: Tunnel up max(testingbot\_tunnel\_up) Active connections sum(testingbot\_active\_connections) 2xx request rate rate(testingbot\_http\_requests\_total{code=~"2.."}[5m]) 5xx error rate rate(testingbot\_http\_requests\_total{code=~"5.."}[5m]) p95 HTTP latency histogram\_quantile(0.95, sum by (le) (rate(testingbot\_http\_request\_duration\_seconds\_bucket[5m]))) Bytes throughput rate(testingbot\_proxy\_bytes\_transferred\_total[5m]) Reconnects sum(testingbot\_tunnel\_reconnects\_total) JVM heap usage sum by (area) (jvm\_memory\_bytes\_used) Process CPU rate(process\_cpu\_seconds\_total[5m]) Threads jvm\_threads\_current ## Alert ideas - **Tunnel restart:** fire when `testingbot_uptime_seconds` drops to under 60 seconds. - **Error spike:** fire when the 5-minute error rate exceeds your normal baseline. - **Stalled connections:** fire when `testingbot_connections` stays at zero during normal CI hours. - **Endpoint unreachable:** fire on Prometheus `up == 0` for the tunnel target. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/tunnel/upgrading # Upgrading from 4.x to 5.0 Tunnel 5.0 adds 28 options and removes none, so your existing command lines keep working. Six things do change. The first three are the ones most people notice. - v5.0 - No options removed - Java 17 ## What changes | Area | 4.x | 5.0 | To keep 4.x behaviour | | --- | --- | --- | --- | | **Java** | 11 | **17** | None: hard requirement, or use a [self-contained build](https://testingbot.com/support/tunnel/installation#standalone) | | **What listeners bind** | every interface | `127.0.0.1` | `--bind-address 0.0.0.0` | | **Per-request logging** | one line per request | only failures and 5xx | `--log-http url` | | **Selenium relay logging** | always logged | honours `--log-http` | `--log-http forwarder:url` | | **Docker image** | n/a | sets `TESTINGBOT_BIND_ADDRESS=0.0.0.0` | None: published ports keep working | | **Embedding the JAR** | Jetty 11 + Servlet API | Jetty 12 core handlers | Source change, see [below](https://testingbot.com#embedding) | ## Java 17 is required Tunnel 5.0 is built on Jetty 12, which needs Java 17. The JAR's classes cannot be loaded by an older JVM. On an older Java the tunnel now says so plainly, instead of failing with `A JNI error has occurred`. Check what you have: ```bash java -version ``` If you cannot upgrade Java on the machine, download the [self-contained build](https://testingbot.com/support/tunnel/installation#standalone) for Linux, macOS or Windows. It bundles a trimmed Java runtime, so nothing has to be installed. The [Docker image](https://testingbot.com/support/tunnel/installation#docker) already ships Java 17. ## Listeners bind loopback In 4.x, the Selenium relay (`4445`), the local proxy (`8087`), the metrics port (`8003`) and `--web` (`8080`) accepted connections from any machine that could route to yours. None of those listeners authenticates. The Selenium relay attaches your TestingBot key and secret to everything it forwards, and the local proxy will connect anywhere your machine can, including its own loopback. In 5.0 they bind `127.0.0.1`. Same machine Nothing changes Tests that connect to `localhost:4445` on the tunnel machine work as before. This is the normal setup. Different machine Add `--bind-address 0.0.0.0` Then restrict who can reach the ports with a firewall rule. ```bash java -jar testingbot-tunnel.jar --bind-address 0.0.0.0 ``` Docker The image sets `TESTINGBOT_BIND_ADDRESS=0.0.0.0` for you, because a loopback bind inside a container makes published ports unreachable. Narrow access on the host side of the publish instead: `-p 127.0.0.1:4445:4445`. ## Successful requests no longer log `--log-http` now defaults to `errors`: the request line and headers are logged only for failed or 5xx responses. The Selenium relay follows the same setting, where it previously logged every request. ```bash # one line per request, as in 4.x java -jar testingbot-tunnel.jar --log-http url # browser traffic quiet, Selenium relay verbose java -jar testingbot-tunnel.jar --log-http proxy:none,forwarder:url ``` See the [logging guide](https://testingbot.com/support/tunnel/logging) for per-module levels and JSON output. ## Readiness is stricter `/readyz` reports ready only after the startup self-test passes, and after a reconnect only once both forwarding directions are back. A half-working tunnel that used to look healthy now returns `503` until it can actually carry traffic. A self-test that failed for a transient reason is retried while the SSH connection stays up. If your orchestration waits on `/readyz`, nothing needs to change, it just becomes more accurate. See [health checks](https://testingbot.com/support/tunnel/health-checks). ## Embedding the JAR Only relevant if you start the tunnel from your own Java code rather than as a process. 5.0 is built on Jetty 12's core Handler API with no Servlet layer, so code that reached into the tunnel's Jetty internals needs a source change. The supported surface is `boot()`, `awaitReady(Duration)` and `stop()`, described in the [embedding guide](https://testingbot.com/support/tunnel/embedding). ## Upgrade checklist 1. Install Java 17+, or switch to a [self-contained build](https://testingbot.com/support/tunnel/installation#standalone) or `testingbot/tunnel:5.0`. 2. If tests connect from another host, add `--bind-address 0.0.0.0` and firewall the ports. 3. If you parse the tunnel log per request, add `--log-http url`. 4. Bump Maven to `com.testingbot:TestingBotTunnel:5.0` if you embed the tunnel. 5. Run `--doctor` with your usual flags. In 5.0 it diagnoses the configuration it is actually given. ```bash java -jar testingbot-tunnel.jar --doctor ``` Then look at what 5.0 adds: [traffic rules](https://testingbot.com/support/tunnel/traffic-rules), [config files](https://testingbot.com/support/tunnel/configuration), [Kerberos proxies](https://testingbot.com/support/tunnel/authentication) and [SSH host key pinning](https://testingbot.com/support/tunnel/security#host-key). The full list is in the [changelog](https://testingbot.com/support/tunnel/changelog). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/tunnel/configuration # Flags, files and environment Tunnel options can be set three ways: on the command line, in a `--config` file, or as a `TESTINGBOT_*` environment variable. That lets a container or CI job be configured without assembling a long command line. - Since v5.0 ## Three ways, one order When an option is set in more than one place, the **command line** wins, then the **config file** , then the **environment**. ```bash # these three are equivalent java -jar testingbot-tunnel.jar --se-port 4446 echo "se-port = 4446" > tunnel.conf java -jar testingbot-tunnel.jar --config tunnel.conf TESTINGBOT_SE_PORT=4446 java -jar testingbot-tunnel.jar ``` The environment comes last because it is the most ambient of the three, and the least likely to be what someone meant when they also typed a flag or wrote a file. ## Config file A properties file. Keys are the long option names without the leading dashes. Lines starting with `#` are comments. ```properties # testingbot-tunnel.conf # Run with: java -jar testingbot-tunnel.jar --config testingbot-tunnel.conf tunnel-identifier = staging se-port = 4445 metrics-port = 8003 # Upstream proxy: host:port (HTTP), http://host:port or socks5://host:port proxy = corp-proxy.internal:8080 # Only reach these hosts; everything else gets 403 allow-hosts = staging.example.com,*.internal.example # Domains to refuse (comma separated regular expressions). # Backslashes must be doubled in this file. fast-fail-regexps = .*\\.doubleclick\\.net,.*google-analytics\\.com # Boolean options: true to enable, omit or false to leave off shared = true nobump = false ``` - Boolean options accept `true`, `yes`, `on` or `1`. Anything else, including `false`, leaves the option off. - A boolean set to `false` in the file is not overridden by the environment, so `shared = false` stays off even if `TESTINGBOT_SHARED=true` is set. - The file is read as a Java properties file, so a backslash is an escape character. Write a regular expression's `\.` as `\\.`, or it reaches the tunnel as a bare `.`. - An unknown key stops the tunnel at startup rather than being ignored, so a typo does not silently drop a setting. - Anything given explicitly on the command line still wins over the file. - Run `--help` to list every option, or see the [CLI reference](https://testingbot.com/support/tunnel/commandline). When to use a file A file is the easiest way to manage long `--allow-hosts` or `--fast-fail-regexps` lists, to keep a tunnel's setup in version control, and to share one setup between developers. Keep credentials out of it: use the environment for those. ## Environment variables Long options have a `TESTINGBOT_*` alias, derived from its name: upper-case it, replace dashes with underscores and prefix `TESTINGBOT_`. The few exceptions are listed below the table. | Option | Environment variable | | --- | --- | | --se-port | `TESTINGBOT_SE_PORT` | | --tunnel-identifier | `TESTINGBOT_TUNNEL_IDENTIFIER` | | --allow-hosts | `TESTINGBOT_ALLOW_HOSTS` | | --bind-address | `TESTINGBOT_BIND_ADDRESS` | | --log-http | `TESTINGBOT_LOG_HTTP` | | --shared | `TESTINGBOT_SHARED=true` | A few options have no environment alias because they make no sense on every run: `--help`, `--version`, `--doctor`, `--ready` and `--config`. `TESTINGBOT_AUTH` is special: it takes a comma-separated list of `host:port:user:password` entries, one per `--auth`. This is what makes the [Docker image](https://testingbot.com/support/tunnel/installation#docker) configurable from a compose file or a Kubernetes manifest: ```bash docker run --rm \ -e TESTINGBOT_KEY=... -e TESTINGBOT_SECRET=... \ -e TESTINGBOT_TUNNEL_IDENTIFIER=ci-$BUILD_ID \ -e TESTINGBOT_ALLOW_HOSTS=staging.example.com \ testingbot/tunnel:5.0 ``` ## Where the tunnel reads your key and secret - Positional arguments: `java -jar testingbot-tunnel.jar KEY SECRET` - `client-key` and `client-secret` in a `--config` file - The `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` environment variables - A `~/.testingbot` file containing `key:secret` Get your key and secret from your [account page](https://testingbot.com/members/user/edit). ## Keeping credentials out of the process list Anything on the command line is visible to other users of the machine in `ps`. Options that carry a secret are better set in the environment. | Option | Environment variable | Format | | --- | --- | --- | | KEY SECRET | `TESTINGBOT_KEY`, `TESTINGBOT_SECRET` | your account credentials | | --auth | `TESTINGBOT_AUTH` | `host:port:user:password`, comma-separated for several | | --proxy-userpwd | `TESTINGBOT_PROXY_USERPWD` | `user:password` | | --proxy-testingbot-userpwd | `TESTINGBOT_PROXY_TESTINGBOT_USERPWD` | `user:password` | | --metrics-auth | `TESTINGBOT_METRICS_AUTH` | `user:password` | Positional credentials The positional `KEY SECRET` form is still supported, but it puts your secret in `ps` for as long as the tunnel runs. On a shared machine or CI runner, prefer `TESTINGBOT_KEY` / `TESTINGBOT_SECRET` or `~/.testingbot`. A flag always wins over the matching variable. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/tunnel/authentication # Credentials the tunnel presents for you Some of what a test needs to reach sits behind a login prompt, and on corporate networks the route out does too. The tunnel can authenticate on your test's behalf, to **sites** and to **upstream proxies** , with HTTP Basic or Kerberos. - Basic - Kerberos / SPNEGO | | Basic | Kerberos (Negotiate) | | --- | --- | --- | | **Sites** your test opens | `--auth` | `--krb5-hosts` | | **Upstream proxy** for test traffic | `--proxy-userpwd` | `--proxy-auth-scheme negotiate` | | **Proxy to TestingBot** | `--proxy-testingbot-userpwd` | `--proxy-auth-scheme negotiate` | ## Basic auth for protected sites Staging sites often sit behind HTTP Basic authentication. Rather than putting credentials in every test URL, let the tunnel answer the prompt. ```bash java -jar testingbot-tunnel.jar -a staging.example.com:80:myuser:mypassword ``` - The format is `host:port:user:password`. Pass several entries for several sites. - To keep the password out of `ps`, use `TESTINGBOT_AUTH` instead, with entries comma-separated. - This works on plain HTTP. HTTPS reaches the site as an encrypted `CONNECT` tunnel that the local tunnel cannot add headers to. ```bash export TESTINGBOT_AUTH=staging.example.com:80:myuser:mypassword,admin.example.com:80:ops:secret java -jar testingbot-tunnel.jar ``` ## Basic auth for upstream proxies ```bash export TESTINGBOT_PROXY_USERPWD=alice:hunter2 java -jar testingbot-tunnel.jar --proxy proxy.corp:8080 ``` The same credentials are used for SOCKS5 proxies (`--proxy socks5://host:port`). If TestingBot's own traffic leaves through a different proxy, give that one its own credentials with `--proxy-testingbot-userpwd`; see [where credentials go](https://testingbot.com#scoping). ## Kerberos proxies (SPNEGO / Negotiate) Many corporate proxies require Kerberos rather than a username and password. The tunnel speaks Negotiate on every path out: browser traffic, HTTPS CONNECT and its own connection to TestingBot. ```bash kinit alice@CORP.EXAMPLE java -jar testingbot-tunnel.jar --proxy proxy.corp:8080 --proxy-auth-scheme negotiate ``` | Option | Purpose | | --- | --- | | --proxy-auth-scheme \ | How to authenticate to the upstream proxy. Default `basic`. | | --proxy-spn \ | Service principal of the proxy. Defaults to `HTTP/`. Accepts `HTTP/host` or `HTTP@host`. | | --krb5-keytab \ | Keytab to log in from, for machines where nobody runs `kinit`. Requires `--krb5-principal`. | | --krb5-principal \ | Principal to log in as with the keytab, for example `svc-tunnel@CORP.EXAMPLE`. | Without a keytab, credentials come from your existing Kerberos ticket cache. ## Unattended Kerberos with a keytab On a build agent or in a container, nobody is there to run `kinit`. Give the tunnel a keytab for a service account instead. ```bash java -jar testingbot-tunnel.jar \ --proxy proxy.corp:8080 \ --proxy-auth-scheme negotiate \ --krb5-keytab /etc/tunnel/svc-tunnel.keytab \ --krb5-principal svc-tunnel@CORP.EXAMPLE ``` Treat the keytab like a password: it can obtain tickets as that principal. ## Kerberos for intranet sites `--krb5-hosts` sends the same Kerberos credentials to _sites_, for intranet applications that log users in with Kerberos. ```bash java -jar testingbot-tunnel.jar --krb5-hosts intranet.corp.example,wiki.corp.example ``` - It is an explicit list and never a wildcard. A service ticket names the user, so every host that may receive one has to be written down. - Empty by default, so no site receives Kerberos credentials unless you name it. - Plain HTTP only. HTTPS reaches the target as an opaque `CONNECT` tunnel. ## Where credentials go Proxy credentials are only ever sent to the proxy they were issued for, matched on protocol, host and port. - `--proxy-userpwd` goes to `--proxy` and nowhere else. - `--proxy-testingbot` does **not** inherit `--proxy-userpwd`. The two proxies usually belong to different operators, so give it its own `--proxy-testingbot-userpwd`. - A proxy chosen by a [PAC file](https://testingbot.com/support/tunnel/upstream-proxy#pac-local) receives no credentials. The tunnel logs that it withheld them, so the resulting `407` explains itself. ```bash java -jar testingbot-tunnel.jar \ --proxy 10.0.0.9:8080 --proxy-userpwd testuser:testpass \ --proxy-testingbot corp-egress:3128 --proxy-testingbot-userpwd egress:secret ``` ## Checking a setup Every Negotiate misconfiguration looks like the same `407` from the proxy. `--doctor` walks the whole chain and reports exactly which step fails. ```bash java -jar testingbot-tunnel.jar --proxy proxy.corp:8080 --proxy-auth-scheme negotiate --doctor ``` Run it with the same options you start the tunnel with. It checks the configuration it is actually given. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/tunnel/traffic-rules # Decide what the tunnel may reach A tunnel can reach whatever the machine running it can reach. These options narrow that down to what your tests actually need, and steer requests to a different destination without touching your tests. - Since v5.0 - HTTP, CONNECT and WebSocket Enforced on every way out `--allow-hosts`, `--fast-fail-regexps` and `--localhost-policy` apply to plain HTTP requests, HTTPS `CONNECT` tunnels and WebSocket upgrades alike. A refused request gets a `403` that says which rule refused it. ## Allow only named hosts `--allow-hosts` names what the tunnel is _for_. Everything else is refused. ```bash java -jar testingbot-tunnel.jar --allow-hosts 'staging.example.com,*.internal.example' ``` - An entry is an exact host or `*.suffix` for its subdomains. - `*.internal.example` covers `app.internal.example` but not `internal.example` itself. List the apex separately if you need it. - Omit the option and any host is reachable, as in 4.x. This is the safer of the two approaches. A deny list only stops destinations somebody thought of in advance. An allow list keeps a tunnel on a CI runner from becoming a route into everything else on that network. ## Fast-fail domains `--fast-fail-regexps` refuses requests to hosts matching any of a comma-separated list of regular expressions. The browser gets an immediate answer instead of waiting on a request that would go through the tunnel. ```bash # skip analytics and ad networks to speed up page loads java -jar testingbot-tunnel.jar -F '.*google-analytics\.com,.*doubleclick\.net' ``` **New in 5.0:** prefix an entry with `!` to make an exception. This blocks everything except `ok.com` and its subdomains: ```bash java -jar testingbot-tunnel.jar -F '.*,!ok\.com' ``` An exception has to cover a whole host or subdomain, so it cannot be satisfied by a name that merely ends with it: `ok.com.attacker.net` is _not_ excepted by `!ok\.com`. Which one to use To speed tests up by skipping third-party noise, use `--fast-fail-regexps`. To limit what the tunnel may reach, use `--allow-hosts`. You can combine them: a host has to pass both. ## Localhost policy `--localhost-policy deny` refuses tunnel traffic that would reach this machine's own loopback interface. ```bash java -jar testingbot-tunnel.jar --localhost-policy deny ``` Use it for a tunnel that is meant to reach a staging network, and nothing running on the machine itself, such as a shared CI runner or a jump host. The check is applied to the address actually dialled, not only to the name in the URL, so a DNS answer that points a public-looking name at `127.0.0.1` cannot walk past it. The default is `allow`, which is what you want when testing a development server on your laptop. The Selenium relay is not affected. For how cloud browsers reach your local server, see [Can I use localhost?](https://testingbot.com/support/tunnel/faq#localhost) ## Remap destinations with --connect-to Dial a different host or port than the request asks for, while leaving the URL, the `Host` header and the TLS SNI name untouched. The format matches curl's `--connect-to`: `HOST1:PORT1:HOST2:PORT2`. ```bash # test https://www.example.com against the staging instance java -jar testingbot-tunnel.jar --connect-to 'www.example.com:443:staging-lb.internal:443' # send every request on port 80 to a local dev server on 3000 java -jar testingbot-tunnel.jar --connect-to ':80:127.0.0.1:3000' ``` - An empty `HOST1` or `PORT1` matches anything. An empty `HOST2` or `PORT2` leaves that half of the destination unchanged. - Comma-separate several rules. The first rule that matches wins. - Bracket IPv6 literals: `example.com:443:[::1]:8443`. This lets a test drive the production hostname against another instance, with the same virtual host and certificate name the real site uses. Editing `/etc/hosts` is the usual alternative, but it needs root, affects every process on the machine and cannot change a port. ## Custom DNS `--dns` resolves hostnames through servers you name instead of the system resolver. Useful for split-horizon setups where the machine's default resolver does not know internal names. ```bash java -jar testingbot-tunnel.jar --dns 10.0.0.53,10.0.0.54:5353 ``` | Option | Effect | | --- | --- | | --dns \ | Host or `host:port`, comma separated. The first is primary and the rest are tried in order when it does not answer. Falls back to the system resolver if none can. | | --dns-round-robin | Spread queries across the servers instead of treating the first as primary, for a pool whose members are equal. | | --dns-timeout \ | How long to wait for each server. Default `5`. | ## Connection timeouts | Option | Default | Effect | | --- | --- | --- | | --http-dial-timeout \ | 5 for the API, 15 for tunnelled connections | How long to wait for a TCP connection to a target or upstream proxy. Lower it where unreachable hosts should fail fast. Also bounds the SSH control connection. | | --http-idle-timeout \ | 120 for HTTP, 300 for CONNECT | How long a connection may sit with no traffic before it is closed. Raise it for long-lived idle connections such as long polling. | ## Reading refusals and errors When the tunnel cannot or will not complete a request, it says why three ways: the HTTP status, an `X-TestingBot-Error` header, and a one-line explanation in the body that shows up in the cloud browser's window. | X-TestingBot-Error | Status | Meaning | | --- | --- | --- | | not-allowed | 403 | The host is not in `--allow-hosts`. | | denied-by-fast-fail | 403 | Blocked by `--fast-fail-regexps`. | | denied-localhost | 403 | Blocked by `--localhost-policy deny`. | | dns-error | 502 | The hostname could not be resolved from the tunnel machine. | | connection-refused | 502 | Nothing is listening on that port. | | connection-timeout | 504 | Timed out connecting to the target. | | host-unreachable | 502 | No network route to the target from the tunnel machine. | | tls-error | 502 | The TLS handshake with the target failed. | | upstream-proxy-\* | 502 | The [upstream proxy](https://testingbot.com/support/tunnel/upstream-proxy) was unreachable, rejected the credentials, or refused the destination. | | loop-detected | 508 | The request targets the tunnel's own proxy port. | The same reason string is the `reason` label on the proxy error counter in [Prometheus metrics](https://testingbot.com/support/tunnel/monitoring), so a spike on a dashboard and a header in a bug report name the same thing. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/tunnel/ssl-bumping # HTTPS through the tunnel Two different certificate questions come up with a tunnel. Does the **cloud browser** trust the certificate your site presents? And does the **tunnel itself** trust the network it runs on? This page covers both. ## What SSL bumping does By default, HTTPS traffic through the tunnel is intercepted and re-signed with the TestingBot CA. This is called SSL bumping. Every TestingBot VM and device already trusts the TestingBot CA. Because the cloud browser sees a certificate it trusts, staging sites with self-signed or expired certificates load without certificate warnings, with no changes to your servers or tests. Bumping on Default Self-signed, expired and internal-CA certificates just work. Bumping off `--nobump` HTTPS is relayed as opaque bytes. The browser sees your server's real certificate, which it has to trust on its own. ## When to disable bumping ```bash java -jar testingbot-tunnel.jar --nobump ``` - **WebSockets and Server-Sent Events.** Streaming protocols need `--nobump` together with `--nocache`. See the [FAQ](https://testingbot.com/support/tunnel/faq#websockets). - **Certificate pinning** in a mobile app, see below. - **Mutual TLS** , where your server expects a client certificate that a re-signing proxy cannot present. - **Testing the certificate itself** , for example HSTS, certificate transparency or an expiry warning. `--nobump` applies to the whole tunnel. If only one environment needs it, run a separate tunnel for that environment with its own [tunnel identifier](https://testingbot.com/support/tunnel/multiple). ## Certificate pinning An app that pins its server certificate will reject the re-signed certificate. You have two options. Trust the TestingBot CA [Download the certificate](https://testingbot.com/downloads/testingbot-ssl-bumping.crt) and add it to your test build's trust store or pin set. Disable bumping Start the tunnel with `--nobump` so the app sees your server's real certificate. ## Networks that intercept TLS The other direction: some corporate proxies intercept outgoing TLS and re-sign it with an internal certificate authority. Java does not trust that authority, so the tunnel cannot even register itself with TestingBot. Point the tunnel at the authority's certificate, in PEM format. Repeat the option for several. ```bash java -jar testingbot-tunnel.jar --proxy proxy.corp:8080 --cacert-file /etc/ssl/corp-ca.pem ``` - The certificate is **added** to the platform's trust store, not used instead of it, so public sites keep working. - `--doctor` prints the subjects of the certificates it loaded, so you can confirm it picked up the right one. - This affects what the tunnel process trusts. It does not change what the cloud browser trusts. More on corporate networks: [upstream proxies](https://testingbot.com/support/tunnel/upstream-proxy) and [proxy authentication](https://testingbot.com/support/tunnel/authentication). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/tunnel/headers # Edit headers on the way through `--header` edits the headers the tunnel sends to your site. `--response-header` edits the headers your site sends back to the cloud browser. Both are repeatable, and neither needs a change to your application. - Since v5.0 - Plain HTTP ## Rule grammar Each `--header` or `--response-header` takes one rule. Repeat the option for more. | Rule | Effect | | --- | --- | | name: value | Set the header, replacing whatever the peer sent. | | name; | Set the header to an empty value. | | -name | Remove the header. | | -name\* | Remove every header whose name starts with `name`. | ## Request headers Headers sent from the tunnel to your site. ```bash # tell the backend this traffic comes from automated tests java -jar testingbot-tunnel.jar --header 'X-Test-Run: nightly-42' # remove every X-Forwarded-* header before it reaches your server java -jar testingbot-tunnel.jar --header '-X-Forwarded*' ``` ## Response headers Headers sent from your site back to the cloud browser. Useful for dropping a policy that blocks a test page without changing the server. ```bash # a strict CSP blocks a test helper script java -jar testingbot-tunnel.jar --response-header '-Content-Security-Policy' # stop a staging site's HSTS from sticking in the browser java -jar testingbot-tunnel.jar --response-header '-Strict-Transport-Security' # allow the page to be framed by a test harness java -jar testingbot-tunnel.jar --response-header '-X-Frame-Options' ``` ## --extra-headers `--extra-headers` adds request headers from a JSON object. It predates `--header` and still works. ```bash java -jar testingbot-tunnel.jar --extra-headers '{"X-Environment":"staging","X-Tenant":"acme"}' ``` For new setups, prefer `--header`: it can also replace and remove headers, and each rule is a separate option, which is easier to manage in a [config file](https://testingbot.com/support/tunnel/configuration) or CI variable. ## HTTPS and headers Plain HTTP only HTTPS arrives at the local tunnel as a `CONNECT` request and is relayed as opaque, encrypted bytes, so the local tunnel cannot see or change its headers. Header rules, `--extra-headers` and [`--auth`](https://testingbot.com/support/tunnel/authentication#basic-sites) apply to plain HTTP traffic. If you need a header on an HTTPS site in a test, set it from the test itself, for example with your framework's request interception or a browser extension. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/tunnel/health-checks # Know when the tunnel can carry traffic Starting the tunnel process is not the same as the tunnel being able to forward requests. A remote tunnel server has to be provisioned, the SSH connection established and a self-test passed. Health checks let your CI job or orchestrator wait for exactly that moment, and notice when it is lost. - Since v5.0 - Docker HEALTHCHECK - Kubernetes probes ## Endpoints Served on the metrics port, `8003` by default (change with `--metrics-port`). | Path | Answers | Use it for | | --- | --- | --- | | /healthz | `200 {"status":"ok"}` whenever the process is answering, including mid-reconnect | Liveness: restart the tunnel only if this fails | | /readyz | `200` when the tunnel is forwarding, `503` otherwise | Readiness: start tests, or route traffic, only when this is 200 | | / | JSON status: version, uptime, requests, bytes transferred | Dashboards and quick inspection | | /metrics | Prometheus exposition | [Monitoring](https://testingbot.com/support/tunnel/monitoring) | ```bash curl -i http://127.0.0.1:8003/readyz ``` - `/readyz` turns 200 only after the startup self-test passes, and after a reconnect only once both forwarding directions are back. - During a reconnect, `/healthz` stays 200 while `/readyz` returns 503. The tunnel is recovering by itself, so it should be taken out of rotation, not restarted. - The health endpoints are deliberately not behind `--metrics-auth`. Container probes have no good way to carry credentials, and the endpoints disclose nothing beyond up or down. - Like every local listener, the metrics port binds `127.0.0.1` unless you set `--bind-address 0.0.0.0`. ## The --ready command Asks a running tunnel whether it is ready, then exits `0` (ready) or `1` (not ready, or nothing listening). No `curl` or `wget` needed. ```bash java -jar testingbot-tunnel.jar --ready echo $? # 0 when ready # a tunnel started with --metrics-port 9003 java -jar testingbot-tunnel.jar --ready --metrics-port 9003 ``` `--ready` queries `/readyz` on `--metrics-port`, so pass the same port you started the tunnel with. ## Waiting for the tunnel in CI Start the tunnel in the background, then block until it is ready before running tests. ```bash java -jar testingbot-tunnel.jar & TUNNEL_PID=$! for i in $(seq 1 60); do java -jar testingbot-tunnel.jar --ready 2>/dev/null && break sleep 2 done java -jar testingbot-tunnel.jar --ready || { echo "tunnel not ready"; exit 1; } npm test kill $TUNNEL_PID ``` On GitHub Actions the [testingbot-tunnel-action](https://testingbot.com/support/integrations/ci-cd/github-actions) does this for you. ## Docker The `testingbot/tunnel` image has a built-in `HEALTHCHECK` that runs `--ready`, so `docker ps` shows _healthy_ once the tunnel is actually forwarding. ```bash docker run -d --name tb-tunnel \ -e TESTINGBOT_KEY=... -e TESTINGBOT_SECRET=... \ -p 127.0.0.1:4445:4445 \ testingbot/tunnel:5.0 docker inspect --format '{{.State.Health.Status}}' tb-tunnel ``` With Docker Compose, other services can wait for it: ```yaml services: tunnel: image: testingbot/tunnel:5.0 environment: TESTINGBOT_KEY: ${TESTINGBOT_KEY} TESTINGBOT_SECRET: ${TESTINGBOT_SECRET} tests: build: . environment: SELENIUM_URL: http://tunnel:4445/wd/hub depends_on: tunnel: condition: service_healthy ``` Changing the metrics port The image's `HEALTHCHECK` assumes the default metrics port. If you set `--metrics-port` or `TESTINGBOT_METRICS_PORT`, override the healthcheck to match. ## Kubernetes Use `/healthz` as the liveness probe and `/readyz` as the readiness probe. Tunnel options are available as `TESTINGBOT_*` environment variables, so no command line is needed. ```yaml apiVersion: apps/v1 kind: Deployment metadata: name: testingbot-tunnel spec: replicas: 1 selector: matchLabels: { app: testingbot-tunnel } template: metadata: labels: { app: testingbot-tunnel } spec: containers: - name: tunnel image: testingbot/tunnel:5.0 env: - name: TESTINGBOT_KEY valueFrom: { secretKeyRef: { name: testingbot, key: key } } - name: TESTINGBOT_SECRET valueFrom: { secretKeyRef: { name: testingbot, key: secret } } - name: TESTINGBOT_TUNNEL_IDENTIFIER value: k8s-staging ports: - containerPort: 4445 # Selenium relay - containerPort: 8003 # metrics and health startupProbe: httpGet: { path: /readyz, port: 8003 } periodSeconds: 5 failureThreshold: 36 readinessProbe: httpGet: { path: /readyz, port: 8003 } periodSeconds: 10 livenessProbe: httpGet: { path: /healthz, port: 8003 } periodSeconds: 30 ``` Who can reach port 4445 The image binds every interface so that the Service can reach it. The Selenium relay attaches your TestingBot credentials to everything it forwards, so restrict access to it with a `NetworkPolicy` and do not expose it outside the cluster. Running more than one tunnel, for example one per environment? See [multiple tunnels](https://testingbot.com/support/tunnel/multiple) for how identifiers route tests to the right one. ## --readyfile `--readyfile FILE` touches a file once the tunnel is ready. It still works, but it cannot express “was ready, has since dropped”. The file is created once and not removed while the process lives, so a tunnel that lost its SSH connection still looks ready on disk. Prefer `--ready` or `/readyz`, which ask the process for its current state. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/tunnel/logging # Quiet when it works, detailed when it fails By default the tunnel logs only requests that fail or return a 5xx, with their headers. That keeps the log readable in normal use and self-diagnosing when a test breaks. When you need more, turn up browser traffic and the Selenium relay independently. - Since v5.0 ## Where logs go and how much | Option | Effect | | --- | --- | | -l,--logfile \ | Also write the log to a file. | | --log-level \ | `error`, `warn`, `info` (default), `debug` or `trace`. | | -d,--debug | Alias for `--log-level debug`. | | --log-http \ | Per-request HTTP logging, see below. | | --log-format \ | Plain text (default) or one JSON object per record. | | --request-id-header \ | Header carrying the correlation id. Default `X-Request-Id`. | ```bash java -jar testingbot-tunnel.jar --logfile tunnel.log --log-level debug ``` ## HTTP request logging `--log-http` controls how much of each proxied request is written to the log. | Mode | Logs | | --- | --- | | errors | **Default.** Request line and headers, only for failed or 5xx responses. | | none | Nothing per request. | | url | One line per request. This was the 4.x behaviour. | | headers | Every request with its headers. | ```bash java -jar testingbot-tunnel.jar --log-http url ``` ## Browser traffic and the Selenium relay, separately The tunnel carries two kinds of traffic: the **proxy** (requests the cloud browser makes to your sites) and the **forwarder** (the Selenium relay on port 4445 that your test talks to). Set a level for each with `proxy:` and `forwarder:`. ```bash # debugging a WebDriver problem: silence page traffic, show every command java -jar testingbot-tunnel.jar --log-http proxy:none,forwarder:headers # debugging a page that will not load: the opposite java -jar testingbot-tunnel.jar --log-http proxy:headers,forwarder:none ``` `forwarder:body` also logs the relay's request body. That body is WebDriver capabilities, which routinely carry your access key, so it is redacted before it is written (see [redaction](https://testingbot.com#redaction)). `proxy:body` is refused: browser traffic has to stream and cannot be buffered for logging. ## Request IDs Every request carries a correlation id. It is printed in brackets on each log line and passed to your server in the `X-Request-Id` header. If the incoming request already has that header, its value is reused. That lets you match a line in the tunnel log to a line in your application's access log. Use a different header name if your stack expects one: ```bash java -jar testingbot-tunnel.jar --request-id-header X-Correlation-Id ``` ## JSON output `--log-format json` writes one JSON object per record, so a log collector does not have to guess where a multi-line message or a stack trace ends. ```bash java -jar testingbot-tunnel.jar --log-format json --logfile tunnel.jsonl ``` Useful when the tunnel runs in Docker or Kubernetes and its output is shipped to Loki, Elasticsearch, Datadog or CloudWatch. ## What is redacted Tunnel logs often end up attached to support tickets and CI artifacts, so credentials are removed before anything is written. - Header values that carry credentials, such as `Authorization`, `Proxy-Authorization`, cookies and `TB-Credentials`. - Credentials in URLs: the `user:password@` part and credential-shaped query values. - In `forwarder:body`, values whose key names look like credentials. A body that cannot be parsed, because of an unknown content type or because it is too large, is described rather than printed. Sharing a log with support? Include the output of `--doctor` as well, it usually saves a round trip. See [troubleshooting](https://testingbot.com/support/tunnel/troubleshooting). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/tunnel/embedding # Run the tunnel inside your JVM The tunnel runs as a library as well as a process. Start it from a JUnit or TestNG fixture, wait until it can carry traffic, and stop it when the suite ends. No separate process to manage, no shell script in CI. - Since v5.0 - Java 17+ ## Dependency ```xml com.testingbot TestingBotTunnel 5.0 test ``` Use the regular artifact, not the shaded JAR `com.testingbot:TestingBotTunnel:5.0` is the library artifact: Jetty, Apache HttpClient, JSch and its other dependencies are declared in its POM, so Maven or Gradle resolves them alongside your own and can settle version conflicts. The downloadable `testingbot-tunnel.jar` (the `shaded` classifier on Maven Central) is built to run on its own: it bundles those libraries under their original package names, so on your classpath they would duplicate, and possibly clash with, the versions your project already uses. ## Lifecycle Configure an `App` with the same settings the command line takes, then `boot()`, `awaitReady()` and `stop()`. ```java import com.testingbot.tunnel.App; import java.time.Duration; App tunnel = new App(); tunnel.setClientKey(System.getenv("TESTINGBOT_KEY")); tunnel.setClientSecret(System.getenv("TESTINGBOT_SECRET")); tunnel.setSeleniumPort(4445); tunnel.setJettyPort(8087); // local proxy, --localproxy tunnel.setMetricsPort(8003); try { tunnel.boot(); // throws TunnelFailedException instead of exiting the JVM if (tunnel.awaitReady(Duration.ofMinutes(2))) { // run the tests } else { throw new IllegalStateException("tunnel not ready: " + tunnel.getSetupFailure()); } } finally { tunnel.stop(); } ``` | Method | What it does | | --- | --- | | boot() | Creates the tunnel. Returns once it has been _created_, which is not the same as it being able to carry traffic: if the tunnel server is still starting, the rest happens in the background. | | awaitReady(Duration) | Waits for the same state `/readyz` reports. Returns `false` as soon as setup gives up, so you do not wait out the timeout for a tunnel that has already failed. | | getSetupFailure() | Why setup failed, for your error message. | | isReady() | The current readiness, without waiting. | | stop() | Shuts the tunnel down and releases everything it held. | Most command-line options have a matching setter, for example `setTunnelIdentifier`, `setShared`, `setBindAddress` and `setProxy`. ## One tunnel per test class ```java import com.testingbot.tunnel.App; import java.time.Duration; import org.junit.jupiter.api.AfterAll; import org.junit.jupiter.api.BeforeAll; class CheckoutIT { static App tunnel; @BeforeAll static void startTunnel() throws Exception { tunnel = new App(); tunnel.setClientKey(System.getenv("TESTINGBOT_KEY")); tunnel.setClientSecret(System.getenv("TESTINGBOT_SECRET")); tunnel.setTunnelIdentifier("checkout-it"); tunnel.boot(); if (!tunnel.awaitReady(Duration.ofMinutes(2))) { throw new IllegalStateException("tunnel not ready: " + tunnel.getSetupFailure()); } } @AfterAll static void stopTunnel() { if (tunnel != null) tunnel.stop(); } // tests use http://localhost:4445/wd/hub with tunnel-identifier "checkout-it" } ``` ## What you can rely on - **Nothing calls `System.exit`.** Every failure while booting throws `TunnelFailedException`. A failure in the background, where there is nobody to throw to, releases everything and leaves a stopped tunnel whose `isReady()` and `/readyz` say so. - **`stop()` gives everything back.** The Selenium relay, local proxy and metrics port are unbound, the SSH session and its timers are cancelled, shutdown hooks are unregistered, the ready file is removed, and the JVM's default `Authenticator` is put back the way it was found. Starting a tunnel per job does not leak. - **Readiness is per tunnel.** Two tunnels in one JVM each answer for themselves, on their own metrics port. - **Your logging is left alone.** Console handlers, log levels and `--log-format` are set up by the command-line entry point, not by `boot()`. ## Shared within a JVM Three things are process-wide, because what they are built on is. - The Prometheus registry behind `/metrics` and the request and byte counters on `/`, which add up across tunnels in the same JVM. - `Authenticator.setDefault`, which the JDK uses for proxy and SOCKS credentials. The tunnel's authenticator answers only for the proxy it was given and delegates everything else to whatever was installed before it. - Uptime, measured from the first tunnel to boot. Not on the JVM? The [NodeJS launcher](https://testingbot.com/support/tunnel/installation#node) and the [GitHub Action](https://testingbot.com/support/integrations/ci-cd/github-actions) manage the tunnel process for you. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/tunnel/commandline # Every flag, every variable The complete commandline reference for TestingBot Tunnel. Pass credentials positionally, via environment variables, or from a `.testingbot` file in your `$HOME`. ## Three ways to authenticate Method 1 Positional args `java -jar testingbot-tunnel.jar KEY SECRET` Method 2 Env vars `TESTINGBOT_KEY` + `TESTINGBOT_SECRET` Method 3 .testingbot file In `$HOME`, containing `key:secret`. ## Environment variables Use environment variables to keep sensitive credentials out of your shell history and the system process list. | Variable | Replaces / purpose | | --- | --- | | TESTINGBOT\_KEY | Your TestingBot API key. Replaces the first positional argument. | | TESTINGBOT\_SECRET | Your TestingBot API secret. Replaces the second positional argument. | | TESTINGBOT\_AUTH | Basic-auth credentials in the form `host:port:user:password`. Replaces `-a`. | | TESTINGBOT\_PROXY\_USERPWD | Upstream proxy credentials as `user:password`. Replaces `-z`. | | TESTINGBOT\_METRICS\_AUTH | HTTP-basic credentials for the metrics endpoint as `user:password`. Replaces `--metrics-auth`. | | TESTINGBOT\_PROXY\_TESTINGBOT\_USERPWD | Credentials for `--proxy-testingbot` as `user:password`. | **Since 5.0, long options have an environment alias:** `--se-port` reads `TESTINGBOT_SE_PORT`, `--allow-hosts` reads `TESTINGBOT_ALLOW_HOSTS`, and so on. The exceptions are `--help`, `--version`, `--doctor`, `--ready` and `--config`, which have no alias, and `TESTINGBOT_AUTH`, which takes a comma-separated list of `--auth` entries. Options can also come from a `--config` file. Precedence is command line, then config file, then environment. See [configuration](https://testingbot.com/support/tunnel/configuration). ## All command-line options Run `--help` for the list your version supports. Options marked _new in 5.0_ need tunnel 5.0 or later. | Option | Description | | --- | --- | | -h,--help | Display help text and exit. | | -v,--version | Display the tunnel version and exit. | | --doctor | Run diagnostics with the options given and exit: DNS, connectivity to TestingBot, free ports, upstream proxy, Kerberos and `--cacert-file` subjects. | | --config \ | Read options from a properties file. Keys are long option names without dashes. _New in 5.0._ See [configuration](https://testingbot.com/support/tunnel/configuration). | | -i,--tunnel-identifier \ | Tag the tunnel with an identifier. When you have multiple tunnels, set this identifier in your test capabilities to pick which tunnel handles the test. | | -s,--shared | Share this tunnel with other users on your team. Tunnels are private by default. | | -P,--se-port \ | Local Selenium port your tests connect to. Default `4445`. | | -j,--localproxy \ | Port the local proxy listens on. Default `8087`. | | --bind-address \ | `127.0.0.1` (default) or `0.0.0.0`. Interface for every local listener: the Selenium relay, local proxy, metrics port and `--web`. _New in 5.0;_ 4.x bound every interface. See [upgrading](https://testingbot.com/support/tunnel/upgrading#bind). | | -p,--hubport \ | Connect to port 80 on the TestingBot hub instead of the default port 4444. | | -x,--noproxy | Do not start the local proxy. Requires you to run your own proxy on port 8087. | | -q,--nocache | Bypass the caching proxy running on the tunnel VM. Required for WebSocket and SSE proxying. | | -w,--web \ | Serve a local directory over a built-in webserver. Handy for static-file testing. | | -f,--readyfile \ | Touch this file when the tunnel is ready. Prefer `--ready` for health checks. | ## Traffic rules, headers and site auth | Option | Description | | --- | --- | | --allow-hosts \ | Only reach these hosts; everything else is refused with 403. `*.suffix` matches subdomains. _New in 5.0._ | | -F,--fast-fail-regexps \ | Comma-separated regular expressions for hosts the tunnel should refuse. Prefix an entry with `!` to make an exception, e.g. `.*,!ok\.com`. | | --localhost-policy \ | Whether tunnel traffic may reach this machine's loopback interface. Default `allow`. _New in 5.0._ | | --connect-to \ | Dial `H2:P2` for requests to `H1:P1`, keeping the URL, Host header and TLS SNI. Comma separated. _New in 5.0._ | | --dns \ | Resolve hostnames with these DNS servers, `host` or `host:port`. The first is primary, the rest are fallbacks. | | --dns-round-robin | Spread queries across the `--dns` servers instead of using the first as primary. | | --dns-timeout \ | Timeout per DNS server. Default `5`. | | --http-dial-timeout \ | TCP connect timeout to targets and upstream proxies. Default 5 for the API, 15 for tunnelled connections. | | --http-idle-timeout \ | Close connections idle this long. Default 120 for HTTP, 300 for CONNECT. | | -a,--auth \ | Perform HTTP Basic Authentication for specific hosts. Plain HTTP only. See [authentication](https://testingbot.com/support/tunnel/authentication). | | --krb5-hosts \ | Send Kerberos (SPNEGO) credentials to these sites. Never wildcarded. Plain HTTP only. | | --header \ | Edit a request header: `name: value` sets, `name;` empties, `-name` removes, `-name*` removes by prefix. Repeatable. Plain HTTP only. See [header rewriting](https://testingbot.com/support/tunnel/headers). | | --response-header \ | Edit a response header, same grammar as `--header`. For example `-Content-Security-Policy`. | | --extra-headers \ | Add request headers from a JSON object. | | --pac \ | PAC URL for the **remote browser** to use. Not the same as `--pac-local`. | ## Upstream proxy | Option | Description | | --- | --- | | -Y,--proxy \<[scheme://]host:port\> | Upstream proxy for egress: `host:port`, `http://host:port` or `socks5://host:port`. Also used for the connection to TestingBot unless `--proxy-testingbot` is set. | | -z,--proxy-userpwd \ | Credentials for `--proxy`. Env: `TESTINGBOT_PROXY_USERPWD`. | | --proxy-auth-scheme \ | How to authenticate to the upstream proxy. `negotiate` is Kerberos/SPNEGO. | | --proxy-spn \ | Service principal for Negotiate. Default `HTTP/`. | | --krb5-keytab \ | Keytab for unattended Kerberos. Requires `--krb5-principal`. | | --krb5-principal \ | Principal to log in as with the keytab. | | --proxy-testingbot \ | Separate proxy for reaching TestingBot itself: the API and the SSH connection. | | --proxy-testingbot-userpwd \ | Credentials for `--proxy-testingbot`. Not inherited from `--proxy-userpwd`. | | --pac-local \ | Choose this tunnel's egress per destination from a PAC file. A plain `http://` URL needs `--pac-local-sha256`. See [PAC files](https://testingbot.com/support/tunnel/upstream-proxy#pac-local). | | --pac-local-sha256 \ | SHA-256 the PAC document must have. | | --pac-test \ | Evaluate `--pac-local` for one URL, print the result and exit. | | --ws-proxy-mode \ | How a `ws://` upgrade traverses `--proxy`. Default `connect`. | ## TLS and SSH | Option | Description | | --- | --- | | -b,--nobump | Do not perform SSL bumping. Required for WebSocket and SSE proxying. See [SSL bumping](https://testingbot.com/support/tunnel/ssl-bumping). | | --cacert-file \ | Trust an additional CA (PEM), for proxies that intercept TLS. Repeatable. Added to the platform trust store. | | --ssh-host-key \ | Pin the SHA-256 fingerprint the tunnel server's SSH host key must have. Repeatable. See [security](https://testingbot.com/support/tunnel/security#host-key). | | --ssh-host-key-policy \ | Without a fingerprint: `warn` (default) connects and says so, `require` refuses. | ## Logging and health | Option | Description | | --- | --- | | -l,--logfile \ | Write logs to a file in addition to stdout. | | --log-level \ | One of `error`, `warn`, `info` (default), `debug`, `trace`. | | -d,--debug | Alias for `--log-level debug`. | | --log-http \ | `none`, `url`, `headers` or `errors` (default). Per module: `proxy:none,forwarder:headers`. See [logging](https://testingbot.com/support/tunnel/logging). | | --log-format \ | One JSON object per record with `json`. | | --request-id-header \ | Correlation id header. Default `X-Request-Id`. | | --metrics-port \ | Port for the JSON status, Prometheus `/metrics`, `/healthz` and `/readyz`. Default `8003`. | | --metrics-auth \ | Protect `/metrics` with HTTP Basic Authentication. Health endpoints stay open. | | --ready | Query a running tunnel's `/readyz` and exit 0 (ready) or 1. See [health checks](https://testingbot.com/support/tunnel/health-checks). | See practical configurations on the [examples](https://testingbot.com/support/tunnel/examples) page, or the full [installation guide](https://testingbot.com/support/tunnel/installation) for Docker, Maven and the NodeJS launcher. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/tunnel/troubleshooting # When something does not look right Most tunnel problems fall into one of four categories: a missing prerequisite, a network or firewall block, a wrong flag combination or a credential mismatch. This page walks through the common symptoms, what causes them, and how to fix them. ## Run --doctor first The built-in health check validates ports, outbound connectivity and credentials in one step. Run it before anything else when something looks wrong. ```bash java -jar testingbot-tunnel.jar --doctor ``` `--doctor` exits with a non-zero status if any check fails, which makes it usable as a CI pre-flight step. ## Issues you may encounter ### Address already in use on port 4445 or 8087 Another process is bound to one of the tunnel's local ports. Identify it with `lsof -i :4445` on macOS or Linux, or `netstat -ano | findstr :4445` on Windows. Stop the offending process or change the tunnel's local ports with `--se-port 4446` and `--localproxy 8088`. ### --doctor reports "Cannot connect to hub.testingbot.com:443" The tunnel cannot reach TestingBot. Verify outbound port 443 to `*.testingbot.com` is open. If you are behind a corporate proxy, route tunnel traffic through it with `--proxy host:port` and `--proxy-userpwd user:password`. See the [upstream proxy guide](https://testingbot.com/support/tunnel/upstream-proxy). ### Authentication failed: 401 Unauthorized Your TestingBot key or secret is wrong, swapped, or has been rotated. Copy them again from [your account settings](https://testingbot.com/members/user/edit). If you use environment variables, make sure `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` are exported in the same shell that runs the tunnel. ### SSL handshake fails inside the cloud browser Your test target uses certificate pinning or a non-standard CA chain. Either [download the TestingBot CA certificate](https://testingbot.com/downloads/testingbot-ssl-bumping.crt) and add it to the trust store on the cloud VM, or disable SSL bumping with the `--nobump` flag. ### WebSocket connection fails or upgrades hang WebSocket and SSE proxying requires both `--nobump` and `--nocache` to be set. The local caching proxy and the SSL-bumping interceptor cannot forward streaming protocols, so they must be disabled. These flags are prerequisites, not optional fix-it switches. ### Tests are unusually slow through the tunnel Bypass non-essential domains with `--fast-fail-regexps`. A typical setting like the one below often halves the proxied request count: ```bash java -jar testingbot-tunnel.jar -F ".*google-analytics\.com.*,.*facebook\.com.*,.*doubleclick\.net.*" ``` If your network has limited bandwidth, run [multiple tunnels in parallel](https://testingbot.com/support/tunnel/multiple) on different machines and let TestingBot load-balance across them. ### Tunnel disconnects every few minutes A firewall or NAT is killing idle connections. Watch the connection counters at `http://localhost:8003` to see reconnect frequency, as covered in the [monitoring guide](https://testingbot.com/support/tunnel/monitoring). If JVM memory is the cause, increase the heap: ```bash java -Xmx2g -jar testingbot-tunnel.jar key secret ``` ### Java version error: "class file has wrong version" TestingBot Tunnel 5.0 requires Java 17 or higher (4.x required Java 11). If you cannot upgrade Java, use a [self-contained build](https://testingbot.com/support/tunnel/installation#standalone), which bundles its own runtime. Otherwise verify with `java -version` and install Java 17 LTS if needed. See the [installation guide](https://testingbot.com/support/tunnel/installation). ### localhost works on desktop VMs but not on mobile devices Mobile simulators, emulators and physical devices cannot resolve the literal hostname `localhost`. Add a custom hostname to `/etc/hosts` on the tunnel host, such as `127.0.0.1 mywebsite.local`, and reference `http://mywebsite.local` in your tests. ### Tunnel shows as active in dashboard after I stopped it The tunnel was terminated abruptly (`kill -9` or hard machine shutdown) and could not deregister itself. Wait up to 5 minutes for the TestingBot grid to time it out, or shut down gracefully next time: press `Ctrl+C`, delete `~/.testingbot-tunnel.pid`, or send `kill -SIGUSR1 `. ## Reading the logs Re-run the tunnel with `--log-level debug` and `--logfile` to capture detailed output to a file. ```bash java -jar testingbot-tunnel.jar --log-level debug --logfile tunnel.log key secret ``` The log file is the first thing support will ask for. `--debug` additionally prints request and response headers, useful when diagnosing application-level issues. ## Contact support If none of the steps above resolve your issue, reach out to [TestingBot support](https://testingbot.com/contact/new) with: - The exact tunnel version (`java -jar testingbot-tunnel.jar --version`). - Your `--doctor` output. - The debug log file from a failing run. - The flags you started the tunnel with. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/tunnel/faq # Common questions, direct answers The most common questions about installing, configuring and operating TestingBot Tunnel. If your question is not answered here, see the [troubleshooting guide](https://testingbot.com/support/tunnel/troubleshooting) or [contact support](https://testingbot.com/contact/new). ## How do I use the tunnel with manual (live) testing? When starting a manual test session, select which tunnel to use from the session configuration screen. This works across all platforms: desktop VMs, simulators, emulators and physical devices. Team members can start shared tunnels (with the `--shared` flag) that everyone on the team can use during manual testing. ## How do I handle SSL certificate pinning in my mobile app? By default, TestingBot Tunnel performs SSL interception (also called SSL bumping) to handle common certificate issues like self-signed or expired certificates. All TestingBot VMs and devices are configured to trust our CA certificate. If your app uses certificate pinning, [download the TestingBot CA certificate](https://testingbot.com/downloads/testingbot-ssl-bumping.crt) and add it to your app's trusted certificates. Alternatively, disable SSL interception by starting the tunnel with the `--nobump` flag. ## How many tunnels can I run simultaneously? The number of simultaneous tunnels depends on your account plan. The default limit is 2 concurrent tunnels. [Upgrade your plan](https://testingbot.com/pricing) if you need more. ## Can I use localhost with the tunnel? **The reliable answer, everywhere:** use `local.testingbot.com` instead of `localhost`. It resolves to `127.0.0.1`, so it reaches exactly the same server on your machine, and it works on desktop VMs, simulators, emulators and physical devices alike. ```bash http://local.testingbot.com:3000 https://local.testingbot.com:8080 ``` Browsers refuse to send `localhost` through a proxy at all — Chrome, Edge and Firefox all treat it as a special name that means “this machine”. Over `https://` that means the request never leaves the browser and never reaches your tunnel, usually showing up as `ERR_SSL_PROTOCOL_ERROR`. Any name that is not `localhost` avoids the problem entirely. **Plain `http://localhost` on desktop VMs:** this does work, on these ports: `80, 443, 3000, 3001, 3030, 3400, 8080`. For additional ports, use the `localHttpPorts` or `localHttpsPorts` capabilities with an array of port numbers. **Mobile devices:** simulators, emulators and physical devices cannot resolve `localhost` at all. Use `local.testingbot.com`, or add your own hostname to the hosts file on the machine running the tunnel: ```bash # Add to /etc/hosts on your tunnel machine 127.0.0.1 mywebsite.local ``` Then use `http://mywebsite.local` in your tests. The tunnel will route requests to your local server. ## Can I use WebSockets or SSE with the tunnel? Yes, TestingBot Tunnel v4 and later supports WebSockets (`ws://` and `wss://`) and Server-Sent Events (SSE). Prerequisite flags WebSocket and SSE proxying **requires** the `--nobump` and `--nocache` flags. These are prerequisites, not optional fix-it switches: the local caching proxy and the SSL-bumping interceptor cannot forward streaming protocols. ```bash java -jar testingbot-tunnel.jar --nocache --nobump ``` Hostname for WebSocket WebSocket connections cannot use `localhost` directly. Use `local.testingbot.com`, which resolves to `127.0.0.1` and needs no setup: ```bash ws://local.testingbot.com:3000 wss://local.testingbot.com:3000 ``` Or create your own hostname in `/etc/hosts` on the machine running the tunnel: ```bash 127.0.0.1 testingbot.com ``` Then connect using `ws://testingbot.com:3000`. ## How can I improve tunnel performance? Use the `--fast-fail-regexps` (or `-F`) option to bypass the tunnel for domains that do not affect your tests, such as analytics, ads or third-party trackers: ```bash java -jar testingbot-tunnel.jar -F ".*google-analytics\.com.*,.*facebook\.com.*" ``` This reduces the number of requests proxied through the tunnel, improving overall test speed. For high-traffic test runs, consider [running multiple tunnels in parallel](https://testingbot.com/support/tunnel/multiple). ## Can I use the same tunnel for both virtual and real devices? Yes, a single tunnel instance works with desktop VMs, simulators, emulators and physical devices. There is no need to start separate tunnels for different device types. ## Does TestingBot Tunnel work behind a corporate firewall or VPN? Yes. The tunnel only needs outbound access to `*.testingbot.com` on port 443 (HTTPS) and port 22 (SSH). It does not require any inbound ports. For corporate networks that force all traffic through an HTTP proxy, route tunnel traffic through the proxy with `--proxy` and `--proxy-userpwd`: ```bash java -jar testingbot-tunnel.jar --proxy corp-proxy.example.com:8080 --proxy-userpwd user:password ``` See the [upstream proxy guide](https://testingbot.com/support/tunnel/upstream-proxy) for PAC and SOCKS variations. ## What ports does TestingBot Tunnel use? | Direction | Port | Purpose | | --- | --- | --- | | Outbound | 443 | HTTPS to `*.testingbot.com` | | Outbound | 22 | SSH tunnel | | Local bind | 4445 | Selenium hub (override `--se-port`) | | Local bind | 8087 | Local proxy (override `--localproxy`) | | Local bind | 8003 | Metrics endpoint (override `--metrics-port`) | No inbound ports are required. ## Is TestingBot Tunnel free? Yes. TestingBot Tunnel is free for all TestingBot accounts including the free trial. It is open source under an MIT license, [source on GitHub](https://github.com/testingbot/Testingbot-Tunnel). ## How do I update TestingBot Tunnel to the latest version? Download the latest JAR from the [tunnel home page](https://testingbot.com/support/tunnel), unzip and replace your existing `testingbot-tunnel.jar`. If you use Docker, run `docker pull testingbot/tunnel`. If you use Maven, bump the `com.testingbot:TestingBotTunnel` version in your `pom.xml`. See the [changelog](https://testingbot.com/support/tunnel/changelog) for the current release notes. Moving from 4.x to 5.0? Version 5.0 needs Java 17, or a [self-contained build](https://testingbot.com/support/tunnel/installation#standalone) that bundles its own runtime, and its local listeners bind `127.0.0.1` by default. Read [upgrading from 4.x](https://testingbot.com/support/tunnel/upgrading) first. ## Why does my tunnel disconnect or time out? The most common causes are: an idle network connection being killed by a corporate firewall or NAT timeout, JVM out of memory on the tunnel host, or a transient outbound connectivity issue. Run the tunnel with `--doctor` to validate ports and credentials, watch the metrics endpoint on port 8003 for connection counters, and check the log file specified by `--logfile` for stack traces. The [troubleshooting guide](https://testingbot.com/support/tunnel/troubleshooting) walks through the most frequent root causes. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/tunnel/changelog # Release history Every released version of TestingBot Tunnel, newest first. New features, improvements and bug fixes. 1. ## v5.0 October 2026 Current - **Breaking:** requires Java 17 (was 11). On an older Java the tunnel now says so plainly. Or use a new [self-contained build](https://testingbot.com/support/tunnel/installation#standalone) that needs no Java - **Breaking:** local listeners (Selenium relay, local proxy, metrics port, `--web`) bind `127.0.0.1` instead of every interface. Use `--bind-address 0.0.0.0` to accept connections from other machines. The Docker image sets this for you - **Breaking:** successful requests no longer log a line. `--log-http` defaults to `errors`; `--log-http url` restores the 4.x behaviour. See [upgrading from 4.x](https://testingbot.com/support/tunnel/upgrading) - New self-contained downloads for Linux x64/arm64, macOS Intel/Apple silicon (signed and notarized) and Windows x64, with a `.sha256` for every file - Built on Jetty 12 - Restrict what the tunnel can reach: `--allow-hosts`, `!` exceptions in `--fast-fail-regexps` and `--localhost-policy deny`, enforced on HTTP, CONNECT and WebSocket upgrades. See [traffic rules](https://testingbot.com/support/tunnel/traffic-rules) - Corporate networks: `--pac-local` to choose egress per destination from a PAC file, Kerberos/SPNEGO with `--proxy-auth-scheme negotiate` and `--krb5-hosts`, a separate `--proxy-testingbot`, `--cacert-file` for TLS-intercepting proxies, SOCKS5 upstream proxies, `--connect-to`, `--ws-proxy-mode`, multi-server `--dns` and `--http-dial-timeout` / `--http-idle-timeout` - Security: pin the tunnel server's SSH host key with `--ssh-host-key` and `--ssh-host-key-policy require`; upstream proxy credentials are only sent to the proxy they belong to; credentials are redacted from logs - Operations: `/healthz` and `/readyz` endpoints, `--ready` for health checks, and a `HEALTHCHECK` in the Docker image. `/readyz` now reports ready only after the startup self-test passes - Logging: `--log-http` per module (`proxy:`, `forwarder:`), `--log-format json` and request-id correlation - Configuration: `--config` file and a `TESTINGBOT_*` environment variable for nearly every option - `--header` and `--response-header` to add, replace or remove headers on plain HTTP - Failed requests now say why, with an `X-TestingBot-Error` header and a readable message - More resilient reconnects, and `--doctor` diagnoses the configuration it is actually given - The tunnel can be embedded as a Java library with `boot()`, `awaitReady()` and `stop()`. See [embedding](https://testingbot.com/support/tunnel/embedding) 2. ## v4.9 August 2026 - Fixed reverse port forwarding on macOS: every request through the tunnel failed with a `502 Bad Gateway` while the tunnel reported itself healthy - The tunnel now verifies at startup and continuously that reverse forwarding can deliver traffic, and reports failures prominently instead of silently ignoring them - A failed startup self-test no longer prints "The Tunnel is ready" - `--log-level debug` no longer writes credentials to the log - Fixed HTTPS CONNECT host handling for `--fast-fail-regexps` matching and logging - Updated dependencies (security fixes) 3. ## v4.8 May 2026 - Added a Prometheus `/metrics` endpoint exposing HTTP request rate, duration and size histograms, HTTPS CONNECT counters, SSH tunnel state and reconnects, active connections, error counters and JVM/process metrics - Added `--metrics-auth user:password` to protect the metrics endpoint with HTTP Basic Authentication - Added `TESTINGBOT_AUTH`, `TESTINGBOT_PROXY_USERPWD` and `TESTINGBOT_METRICS_AUTH` environment variables so credentials stay out of the process list - Sensitive headers are now stripped from debug logging - Improved basic authentication handling and logging - Updated dependencies 4. ## v4.7 March 2026 - Fixed an issue with WebSocket proxying 5. ## v4.6 January 2026 - Fixed an issue with `--debug` 6. ## v4.5 January 2026 - Added `-s`, `--shared` to share a tunnel with colleagues on your team. Tunnels are not shared by default - Updated dependencies - Code improvements 7. ## v4.4 October 2025 - Improved `--proxy` and `--proxy-userpwd` reliability - Updated dependencies 8. ## v4.3 August 2025 - Enhanced `--doctor` diagnostic checks - Improved connection monitoring and automatic retry mechanisms - Updated Jetty and SSH libraries - Dropped support for Java 8 (Java 11+ now required) 9. ## v4.0 December 2024 - Added WebSocket and Server-Sent Events (SSE) support - Added `--nobump` option to disable SSL bumping - Upgraded to Jetty 11 10. ## v3.5 May 2023 - Upgraded secure connection ciphers for improved security - Updated dependencies 11. ## v3.2 February 2021 - Updated dependencies for security and compatibility 12. ## v3.1 February 2020 - Fixed tunnel polling to prevent indefinite connection attempts - Built with Java 8 for broader compatibility - Internal code improvements 13. ## v3.0 September 2019 - Updated Jetty dependencies - Various stability improvements 14. ## v2.9 January 2019 - Various minor fixes to timeout handling and `--doctor` checks 15. ## v2.8 June 2018 - Fixed null pointer exception in connection handler 16. ## v2.7 June 2018 - Added upstream HTTPS proxy support via CONNECT method (`--proxy`) - Renamed `--jettyport` option to `--localport` - Enabled `--nocache` integration with Manual Testing API - Removed hardcoded port restrictions 17. ## v2.6 April 2018 - Added option to inject custom HTTP headers into requests - Added automatic basic authentication for configured hosts - Clarified `--nocache` behavior: disables all HTTP request caching - Removed deprecated `--boost` and `--ssl` options - Fixed incorrect Java version detection 18. ## v2.5 September 2017 - The tunnel now reports its version to TestingBot - Added `--pac` option to configure a PAC (Proxy Auto-Config) URL for test sessions 19. ## v2.4 August 2017 - Added `--tunnel-identifier` to run multiple tunnels on the same account - Added local directory serving for static file testing - Added embedded REST API for tunnel statistics (default port 8003) - Improved logging with configurable output format, file logging, and debug levels - Improved connection error handling with automatic reconnection 20. ## v2.3 October 2016 - Added `--doctor` command to troubleshoot configuration issues - Debug mode now outputs HTTP headers for easier troubleshooting ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/mobile # Mobile Browser Testing with Selenium TestingBot lets you run Selenium WebDriver tests against **mobile Safari on iOS** (iPhone and iPad) and **mobile Chrome on Android** , on both [simulators/emulators](https://testingbot.com/support/web-automate/browsers) and [real physical devices](https://testingbot.com/support/app-automate/devices) in our EU data centre. Mobile sessions are routed through Appium automatically; no separate Appium install needed on your side. Looking for native app testing instead of mobile browsers? See [App Automate](https://testingbot.com/support/app-automate) for Appium, XCUITest, Espresso and Maestro on real devices. ## Configuring capabilities TestingBot's mobile simulators and emulators can be accessed using Appium. You can use the device picker below to select the capabilities you want to use for your tests. ![OS selected](https://testingbot.com/assets/environments/svg/ios-383468addf160fa18d0e431f529420739d7f7d1206175f682fead627d2e99a52.svg) iOS 18.6 › iPhone 16 Loading environments... Please wait while we load the available devices and platforms. W3C Protocol W3C is the newest WebDriver protocol, TestingBot recommends using W3C. JSONWP JSONWP is the legacy protocol which is no longer actively maintained. Java Python NodeJS C# Ruby PHP ```java MutableCapabilities capabilities = new MutableCapabilities(); capabilities.setCapability(CapabilityType.BROWSER_NAME, "safari"); capabilities.setCapability(CapabilityType.BROWSER_VERSION, "18.6"); capabilities.setCapability("appium:deviceName", "iPhone 16"); capabilities.setCapability(CapabilityType.PLATFORM_NAME, "iOS"); ``` You can configure more options by using the [Appium capabilities generator](https://testingbot.com/support/app-automate/appium/capabilities). ## Portrait/Landscape It's possible to rotate the device before and during your test. Please see these examples: **Set orientation before test (capability):** ```ruby caps = { platformName: "iOS", "appium:deviceName": "iPhone 15", "appium:orientation": "LANDSCAPE" # or PORTRAIT } ``` **Rotate during test:** ```java ((AppiumDriver) driver).rotate(ScreenOrientation.LANDSCAPE); ``` ## Specifying Appium Version TestingBot will use the most recent, compatible, Appium version according to the device, OS and version you specify. If you'd like to specify your own Appium version, you can do this with the `appiumVersion` capability. To learn more about the available Appium versions, please see our [Appium versions documentation](https://testingbot.com/support/app-automate/appium/appium-versions). [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) [C#](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) Pass the `appiumVersion` capability inside `tb:options`. The examples below target an iPhone running mobile Safari, for Android Chrome change `platformName` to `"Android"`, `browserName` to `"chrome"`, and pick an Android `deviceName`. ```java XCUITestOptions options = new XCUITestOptions(); options.setPlatformName("iOS"); options.setDeviceName("iPhone 16"); options.setPlatformVersion("18.6"); options.setBrowserName("safari"); Map tbOptions = new HashMap<>(); tbOptions.put("key", "API_KEY"); tbOptions.put("secret", "API_SECRET"); tbOptions.put("appiumVersion", "2.4.1"); options.setCapability("tb:options", tbOptions); IOSDriver driver = new IOSDriver( new URL("https://hub.testingbot.com/wd/hub"), options); ``` ```python from appium import webdriver from appium.options.ios import XCUITestOptions options = XCUITestOptions() options.platform_name = "iOS" options.device_name = "iPhone 16" options.platform_version = "18.6" options.browser_name = "safari" options.set_capability("tb:options", { "key": "API_KEY", "secret": "API_SECRET", "appiumVersion": "2.4.1" }) driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options) ``` ```javascript const { remote } = require('webdriverio'); const driver = await remote({ hostname: 'hub.testingbot.com', port: 443, protocol: 'https', path: '/wd/hub', user: 'API_KEY', key: 'API_SECRET', capabilities: { platformName: 'iOS', browserName: 'safari', 'appium:deviceName': 'iPhone 16', 'appium:platformVersion': '18.6', 'tb:options': { appiumVersion: '2.4.1' } } }); ``` ```csharp using OpenQA.Selenium.Appium; using OpenQA.Selenium.Appium.iOS; var options = new AppiumOptions(); options.PlatformName = "iOS"; options.BrowserName = "safari"; options.AddAdditionalAppiumOption("deviceName", "iPhone 16"); options.AddAdditionalAppiumOption("platformVersion", "18.6"); options.AddAdditionalAppiumOption("tb:options", new Dictionary { ["key"] = "API_KEY", ["secret"] = "API_SECRET", ["appiumVersion"] = "2.4.1" }); var driver = new IOSDriver( new Uri("https://hub.testingbot.com/wd/hub"), options); ``` ```ruby require 'appium_lib' caps = { caps: { platformName: 'iOS', browserName: 'safari', 'appium:deviceName': 'iPhone 16', 'appium:platformVersion': '18.6', 'tb:options' => { 'appiumVersion' => '2.4.1' } }, appium_lib: { server_url: 'https://API_KEY:API_SECRET@hub.testingbot.com/wd/hub' } } driver = Appium::Driver.new(caps).start_driver ``` ```php $capabilities = [ 'platformName' => 'iOS', 'browserName' => 'safari', 'appium:deviceName' => 'iPhone 16', 'appium:platformVersion' => '18.6', 'tb:options' => [ 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'appiumVersion' => '2.4.1', ], ]; $driver = RemoteWebDriver::create( 'https://hub.testingbot.com/wd/hub', $capabilities ); ``` ## Testing internal or staging mobile sites Use the [TestingBot Tunnel](https://testingbot.com/support/tunnel) to run mobile-browser tests against websites that aren't publicly reachable (local dev servers, staging environments behind a VPN, internal QA sites). Start the tunnel with an identifier, then pass `tunnelIdentifier` in your `tb:options` capabilities so the mobile device routes traffic through your tunnel. See the [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for setup details on Java, Docker and command-line installs. ## Next steps - [Mobile browser & OS list](https://testingbot.com/support/web-automate/browsers) — current iOS/Android browsers available on the grid. - [Real device list](https://testingbot.com/support/app-automate/devices) — physical iPhones, iPads and Android phones/tablets. - [Appium capabilities reference](https://testingbot.com/support/app-automate/appium/capabilities) — full list of mobile capabilities supported. - [Appium versions](https://testingbot.com/support/app-automate/appium/appium-versions) — pin a specific Appium release. - [App Automate](https://testingbot.com/support/app-automate) — native mobile-app testing with Appium, XCUITest, Espresso and Maestro. - [Real device testing overview](https://testingbot.com/real-device-testing) — how the real-device cloud works. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/codeless-automation # Codeless Web Testing Codeless Web Automated Testing allows you to run your pre-recorded Selenium tests on the TestingBot cloud grid. - [Add a test](https://testingbot.com/support/web-automate/codeless-automation/add-test) - [Test Suites](https://testingbot.com/support/web-automate/codeless-automation/suites) - [Schedule a test](https://testingbot.com/support/web-automate/codeless-automation/schedule) - [Test alerts](https://testingbot.com/support/web-automate/codeless-automation/alerts) - [Supported commands](https://testingbot.com/support/web-automate/codeless-automation/commands) - [Variables](https://testingbot.com/support/web-automate/codeless-automation/variables) - [FAQ](https://testingbot.com/support/web-automate/codeless-automation/faq) ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/codeless-automation/add-test # Add Codeless Automation test TestingBot can create and run automated web tests against your website on regular intervals and alert you when a test fails. There are 2 options to create a codeless web automation test: - You can [record your actions](https://testingbot.com#record) with Selenium IDE, [upload the steps](https://testingbot.com#upload) and replay the steps on TestingBot. - You can choose to describe tests in natural language and have the [TestingBot AI agent](https://testingbot.com/support/ai/codeless) create and run the test for you. ## Record your actions ![Selenium IDE](https://testingbot.com/assets/support/codeless/selenium_ide-12210f05c24292c51d5f4043a24db5af420323002b5cb2ebc4ea91dff9753ef1.png) ### Installing Selenium IDE To get started, please make sure to [download the Selenium IDE](https://github.com/SeleniumHQ/selenium-ide/releases/). This is an Electron app that will run on Windows and macOS and will allow you to record and playback actions on a website and allow you to save these actions as Selenium scripts (`.side` files). You can also install the package through NPM: `npm install -g selenium-ide` ### Recording a test #### Creating a test ![Selenium IDE: add test](https://testingbot.com/assets/support/codeless/create-07a560acf47ba92d0e585b2f47f9ac754c713647f7f25f9beb43fa0d98da8ece.png) Once you have the Selenium IDE program running, you can create a new testcase by clicking the plus icon (+). Enter a name for your test. Next, open the URL you want to test in the right side of the Selenium IDE. Once the page is loaded, interact with the page. You will see new commands are added to the left sidebar, which means the IDE is recording all your actions. #### Adding verifications ![Selenium IDE: verification](https://testingbot.com/assets/support/codeless/verify-6f55be33262b647b02156c95eadfecce9b77c0712196521c3ad53fecb30abc43.png) For example, if you tap an element on the webpage, you will see a new `Click` command appear. At any given point, you can right-click an element on the page you are testing and add a verification, for example: `Record Wait For Element Present`. ### Saving test Once you've finished recording your test, you can choose to save it as a `.side` file, which is basically a JSON file that describes all the actions and verifications you added during the recording. Choose `Save Project As` and save the file. You can now [upload this file](https://testingbot.com#upload) to TestingBot and instruct TestingBot to run this test on a scheduled interval, running on the browsers & operating systems you want. ## Upload a codeless test Once you have a `.side` file, you can upload it in the [Codeless Test](https://testingbot.com/members/lab) area of TestingBot. Click the upload button in the **Import a Selenium Test** section and upload the file. Once uploaded, you will be redirected to a test configuration page. Here you can configure various options: ![Selenium IDE: steps](https://testingbot.com/assets/support/codeless/steps-ac508430b3982e357b9ba64cd7df0a9e1655eed37a795a0e87cc9495b058fcf8.png) ### Steps We will display the same steps that were recorded before. You have the option to re-order the steps, edit steps or remove steps. ### Browsers A selection of browsers & operating systems will be available to choose from. Select one or more browsers that you want to run your test on. ### Settings You can configure various settings for your test: - Schedule when the test should run; a specific time, interval, day. Or maybe every 15 minutes. You indicate how often the test will run. - Manage timeouts, screen resolution, screenshots and videos. - Daily and hourly reporting of your tests via email. - Automatically retry a failed test and if the retry succeeds, ignore the failed test. - Receive an alert via email or text message when a test fails. - Test from various geographical locations ## Run Test Now that the test has been added to TestingBot, you can choose to run it immediately. Click the **Run test** button on the top right of the edit test page, or from within the test overview page. TestingBot will run the test on the browser(s) you specified and alert you when a test failed. Each test comes with screenshots, a video and generated log files. ## API Reference You can use the [TestingBot API](https://testingbot.com/support/api) to add, edit, remove and [run Codeless Automation tests](https://testingbot.com/support/api/codeless#runlabtest). Use our [integrations](https://testingbot.com/support/integrations) to improve your workflow. For example, configure a [Slack Bot](https://testingbot.com/support/integrations/slack) to notify you when a codeless automation test failed. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/codeless-automation/suites # Add a Test Suite In our TestLab we offer the possibility to upload test-suites (called **Projects** in Selenium IDE) generated by [Selenium IDE](https://www.selenium.dev/selenium-ide/). A test suite is a collection of tests, generated by the Selenium IDE extension. To upload the suite, please upload the exported (`.side`) file: ![Selenium IDE project](https://testingbot.com/assets/support/codeless/suite-8df25faf22e06b8749c6c03d25dab13bfc7db70fc6f448e8c129f8b1850d76da.png) ## Managing Test Suites Once uploaded to our TestLab, you will see page detailing the Project you just uploaded. It will include a list of the tests that this project contains. On this page, you can change on which browser(s) the tests should run, change alert/scheduling options and more. Changing a setting in the project affects all tests for this project. If you like to only change a single test, click through to the test edit page and make the necessary changes there. ![Selenium IDE project](https://testingbot.com/assets/support/codeless/suite-overview-d0e9677d3f9b170369bb49e9fe68746655cfd20d0ae190b8c03e05c9685dbe50.png) ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/codeless-automation/schedule # Schedule a test When you have [added a test](https://testingbot.com/support/web-automate/codeless-automation/add-test), you can specify when we should run the test, and how many times we should run it (interval). ![Schedule a Test](https://testingbot.com/assets/support/codeless/schedule-9a00c28724a6241167c552558a210e4776b1b5914a0202900428537b803012d7.png) You can have your test either run once, every day, every week, repeat every `..` time or use a custom CRON format. Once scheduled, you can specify when you want to be [alerted of failed tests](https://testingbot.com/support/web-automate/codeless-automation/alerts). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/codeless-automation/alerts # Alerts When a test in Codeless Automation fails, we can notify you by sending you an email, Webhook, Push Notification or SMS/text message. You can choose to be alerted immediately, or once a day. ## Alert Types We offer 3 ways to alert you: - Email - SMS/Text message - Webhook to a URL you specify. The message you will receive will contain the URL, number of errors and an error message trace. ## Email Alerts To receive an alert via email, fill in your email address. We will send an email with the failure of the test. You can customize the error message by filling in your own text. We offer these variables to customize your error text: `%test_name%` and `%browser%`. ![Email Alerts](https://testingbot.com/assets/support/codeless/email-cb45d4d824ab7bd052575692ba60ec586173b33d83175e52e5db63bf6e00ae12.png) ## SMS Text Message Fill in your phone number (with country code) to receive an SMS message when a test fails. You can customize the error message by filling in your own text. We offer these variables to customize your error text: `%test_name%` and `%browser%`. There's also the possibility to be alerted when the test recovers. ![SMS Alerts](https://testingbot.com/assets/support/codeless/sms-1365e3a31475f9d62465031a4edbbd3c1b47c5204762b11af65bcca05fb8ff8b.png) ## Webhook We will call a URL you specify whenever a test fails. The URL needs to be publicly available, otherwise TestingBot cannot reach the URL. The URL will receive a POST request from us with the following fields: - `success` (boolean) - `job_id` (int) - internal id - `test_ids[]` (string[]) webdriver sessionIds - `errors[]` (string[]) error messages ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/codeless-automation/commands # Supported Selenium IDE Commands Below is a list of the commands we currently support in our Codeless Automation. ## Browsing - **close** Simulates the user clicking the "close" button in the titlebar of a popup window or tab. - **closeAndWait** Simulates the user clicking the "close" button in the titlebar of a popup window or tab. - **goBack** Simulates the user clicking the "back" button on their browser. - **goBackAndWait** Simulates the user clicking the "back" button on their browser. - **open** (url) Opens an URL in the test frame. This accepts both relative and absolute URLs. The "open" command waits for the page to load before proceeding, ie. the "AndWait" suffix is implicit. Note: The URL must be on the same domain as the runner HTML due to security restrictions in the browser (Same Origin Policy). If you need to open an URL on another domain, use the Selenium Server to start a new browser session on that domain. - **openAndWait** (url) Opens an URL in the test frame. This accepts both relative and absolute URLs. The "open" command waits for the page to load before proceeding, ie. the "AndWait" suffix is implicit. Note: The URL must be on the same domain as the runner HTML due to security restrictions in the browser (Same Origin Policy). If you need to open an URL on another domain, use the Selenium Server to start a new browser session on that domain. - **openWindow** (url - windowID) Opens a popup window (if a window with that ID isn't already open). After opening the window, you'll need to select it using the selectWindow command. This command can also be a useful workaround for bug SEL-339. In some cases, Selenium will be unable to intercept a call to window.open (if the call occurs during or before the "onLoad" event, for example). In those cases, you can force Selenium to notice the open window's name by using the Selenium openWindow command, using an empty (blank) url, like this: openWindow("", "myFunnyWindow"). - **openWindowAndWait** (url - windowID) Opens a popup window (if a window with that ID isn't already open). After opening the window, you'll need to select it using the selectWindow command. This command can also be a useful workaround for bug SEL-339. In some cases, Selenium will be unable to intercept a call to window.open (if the call occurs during or before the "onLoad" event, for example). In those cases, you can force Selenium to notice the open window's name by using the Selenium openWindow command, using an empty (blank) url, like this: openWindow("", "myFunnyWindow"). - **refresh** Simulates the user clicking the "Refresh" button on their browser. - **refreshAndWait** Simulates the user clicking the "Refresh" button on their browser. - **selectFrame** (locator) Selects a frame within the current window. (You may invoke this command multiple times to select nested frames.) To select the parent frame, use "relative=parent" as a locator; to select the top frame, use "relative=top". You can also select a frame by its 0-based index number; select the first frame with "index=0", or the third frame with "index=2". You may also use a DOM expression to identify the frame you want directly, like this: dom=frames["main"].frames["subframe"] - **selectFrameAndWait** (locator) Selects a frame within the current window. (You may invoke this command multiple times to select nested frames.) To select the parent frame, use "relative=parent" as a locator; to select the top frame, use "relative=top". You can also select a frame by its 0-based index number; select the first frame with "index=0", or the third frame with "index=2". You may also use a DOM expression to identify the frame you want directly, like this: dom=frames["main"].frames["subframe"] - **selectWindow** (windowID) Selects a popup window using a window locator; once a popup window has been selected, all commands go to that window. To select the main window again, use null as the target. Window locators provide different ways of specifying the window object: by title, by internal JavaScript "name," or by JavaScript variable. title=My Special Window: Finds the window using the text that appears in the title bar. Be careful; two windows can share the same title. If that happens, this locator will just pick one. name=myWindow: Finds the window using its internal JavaScript "name" property. This is the second parameter "windowName" passed to the JavaScript method window.open(url, windowName, windowFeatures, replaceFlag) (which Selenium intercepts). var=variableName: Some pop-up windows are unnamed (anonymous), but are associated with a JavaScript variable name in the current application window, e.g. "window.foo = window.open(url);". In those cases, you can open the window using "var=foo". If no window locator prefix is provided, we'll try to guess what you mean like this: 1.) if windowID is null, (or the string "null") then it is assumed the user is referring to the original window instantiated by the browser). 2.) if the value of the "windowID" parameter is a JavaScript variable name in the current application window, then it is assumed that this variable contains the return value from a call to the JavaScript window.open() method. 3.) Otherwise, selenium looks in a hash it maintains that maps string names to window "names". 4.) If that fails, we'll try looping over all of the known windows to try to find the appropriate "title". Since "title" is not necessarily unique, this may have unexpected behavior. If you're having trouble figuring out the name of a window that you want to manipulate, look at the Selenium log messages which identify the names of windows created via window.open (and therefore intercepted by Selenium). You will see messages like the following for each window as it is opened: debug: window.open call intercepted; window ID (which you can use with selectWindow()) is "myNewWindow" In some cases, Selenium will be unable to intercept a call to window.open (if the call occurs during or before the "onLoad" event, for example). (This is bug SEL-339.) In those cases, you can force Selenium to notice the open window's name by using the Selenium openWindow command, using an empty (blank) url, like this: openWindow("", "myFunnyWindow"). - **selectWindowAndWait** (windowID) Selects a popup window using a window locator; once a popup window has been selected, all commands go to that window. To select the main window again, use null as the target. Window locators provide different ways of specifying the window object: by title, by internal JavaScript "name," or by JavaScript variable. title=My Special Window: Finds the window using the text that appears in the title bar. Be careful; two windows can share the same title. If that happens, this locator will just pick one. name=myWindow: Finds the window using its internal JavaScript "name" property. This is the second parameter "windowName" passed to the JavaScript method window.open(url, windowName, windowFeatures, replaceFlag) (which Selenium intercepts). var=variableName: Some pop-up windows are unnamed (anonymous), but are associated with a JavaScript variable name in the current application window, e.g. "window.foo = window.open(url);". In those cases, you can open the window using "var=foo". If no window locator prefix is provided, we'll try to guess what you mean like this: 1.) if windowID is null, (or the string "null") then it is assumed the user is referring to the original window instantiated by the browser). 2.) if the value of the "windowID" parameter is a JavaScript variable name in the current application window, then it is assumed that this variable contains the return value from a call to the JavaScript window.open() method. 3.) Otherwise, selenium looks in a hash it maintains that maps string names to window "names". 4.) If that fails, we'll try looping over all of the known windows to try to find the appropriate "title". Since "title" is not necessarily unique, this may have unexpected behavior. If you're having trouble figuring out the name of a window that you want to manipulate, look at the Selenium log messages which identify the names of windows created via window.open (and therefore intercepted by Selenium). You will see messages like the following for each window as it is opened: debug: window.open call intercepted; window ID (which you can use with selectWindow()) is "myNewWindow" In some cases, Selenium will be unable to intercept a call to window.open (if the call occurs during or before the "onLoad" event, for example). (This is bug SEL-339.) In those cases, you can force Selenium to notice the open window's name by using the Selenium openWindow command, using an empty (blank) url, like this: openWindow("", "myFunnyWindow"). - **windowFocus** Gives focus to the currently selected window - **windowFocusAndWait** Gives focus to the currently selected window - **windowMaximize** Resize currently selected window to take up the entire screen - **windowMaximizeAndWait** Resize currently selected window to take up the entire screen ## Clicks - **click** (locator) Clicks on a link, button, checkbox or radio button. If the click action causes a new page to load (like a link usually does), call waitForPageToLoad. - **clickAndWait** (locator) Clicks on a link, button, checkbox or radio button. If the click action causes a new page to load (like a link usually does), call waitForPageToLoad. - **clickAt** (locator - coordString) Clicks on a link, button, checkbox or radio button. If the click action causes a new page to load (like a link usually does), call waitForPageToLoad. - **clickAtAndWait** (locator - coordString) Clicks on a link, button, checkbox or radio button. If the click action causes a new page to load (like a link usually does), call waitForPageToLoad. - **doubleClick** (locator) Double clicks on a link, button, checkbox or radio button. If the double click action causes a new page to load (like a link usually does), call waitForPageToLoad. - **doubleClickAndWait** (locator) Double clicks on a link, button, checkbox or radio button. If the double click action causes a new page to load (like a link usually does), call waitForPageToLoad. - **doubleClickAt** (locator - coordString) Doubleclicks on a link, button, checkbox or radio button. If the action causes a new page to load (like a link usually does), call waitForPageToLoad. - **doubleClickAtAndWait** (locator - coordString) Doubleclicks on a link, button, checkbox or radio button. If the action causes a new page to load (like a link usually does), call waitForPageToLoad. - **dragAndDrop** (locator - movementsString) Drags an element a certain distance and then drops it - **dragAndDropAndWait** (locator - movementsString) Drags an element a certain distance and then drops it - **dragAndDropToObject** (locatorOfObjectToBeDragged - locatorOfDragDestinationObject) Drags an element and drops it on another element - **dragAndDropToObjectAndWait** (locatorOfObjectToBeDragged - locatorOfDragDestinationObject) Drags an element and drops it on another element ## Form Fields - **addSelection** (locator - optionLocator) Add a selection to the set of selected options in a multi-select element using an option locator. @see #doSelect for details of option locators - **addSelectionAndWait** (locator - optionLocator) Add a selection to the set of selected options in a multi-select element using an option locator. @see #doSelect for details of option locators - **check** (locator) Check a toggle-button (checkbox/radio) - **checkAndWait** (locator) Check a toggle-button (checkbox/radio) - **focus** (locator) Move the focus to the specified element; for example, if the element is an input field, move the cursor to that field. - **focusAndWait** (locator) Move the focus to the specified element; for example, if the element is an input field, move the cursor to that field. - **removeAllSelections** (locator) Unselects all of the selected options in a multi-select element. - **removeAllSelectionsAndWait** (locator) Unselects all of the selected options in a multi-select element. - **removeSelection** (locator - optionLocator) Remove a selection from the set of selected options in a multi-select element using an option locator. @see #doSelect for details of option locators - **removeSelectionAndWait** (locator - optionLocator) Remove a selection from the set of selected options in a multi-select element using an option locator. @see #doSelect for details of option locators - **select** (selectLocator - optionLocator) Select an option from a drop-down using an option locator. Option locators provide different ways of specifying options of an HTML Select element (e.g. for selecting a specific option, or for asserting that the selected option satisfies a specification). There are several forms of Select Option Locator. label=labelPattern: matches options based on their labels, i.e. the visible text. (This is the default.) label=regexp:^[Oo]ther value=valuePattern: matches options based on their values. value=other id=id: matches options based on their ids. id=option1 index=index: matches an option based on its index (offset from zero). index=2 If no option locator prefix is provided, the default behaviour is to match on label. - **selectAndWait** (selectLocator - optionLocator) Select an option from a drop-down using an option locator. Option locators provide different ways of specifying options of an HTML Select element (e.g. for selecting a specific option, or for asserting that the selected option satisfies a specification). There are several forms of Select Option Locator. label=labelPattern: matches options based on their labels, i.e. the visible text. (This is the default.) label=regexp:^[Oo]ther value=valuePattern: matches options based on their values. value=other id=id: matches options based on their ids. id=option1 index=index: matches an option based on its index (offset from zero). index=2 If no option locator prefix is provided, the default behaviour is to match on label. - **setCursorPosition** (locator - position) Moves the text cursor to the specified position in the given input element or textarea. This method will fail if the specified element isn't an input element or textarea. - **setCursorPositionAndWait** (locator - position) Moves the text cursor to the specified position in the given input element or textarea. This method will fail if the specified element isn't an input element or textarea. - **submit** (formLocator) Submit the specified form. This is particularly useful for forms without submit buttons, e.g. single-input "Search" forms. - **submitAndWait** (formLocator) Submit the specified form. This is particularly useful for forms without submit buttons, e.g. single-input "Search" forms. - **uncheck** (locator) Uncheck a toggle-button (checkbox/radio) - **uncheckAndWait** (locator) Uncheck a toggle-button (checkbox/radio) - **getSelectedLabels** (selectLocator) Gets all option labels (visible text) for selected options in the specified select or multi-select element. - **getSelectedLabel** (selectLocator) Gets option label (visible text) for selected option in the specified select element. - **getSelectedValues** (selectLocator) Gets all option values (value attributes) for selected options in the specified select or multi-select element. - **getSelectedValue** (selectLocator) Gets option value (value attribute) for selected option in the specified select element. - **getSelectedIndexes** (selectLocator) Gets all option indexes (option number, starting at 0) for selected options in the specified select or multi-select element. - **getSelectedIndex** (selectLocator) Gets option index (option number, starting at 0) for selected option in the specified select element. - **getSelectedIds** (selectLocator) Gets all option element IDs for selected options in the specified select or multi-select element. - **getSelectedId** (selectLocator) Gets option element ID for selected option in the specified select element. - **verifyAllButtons** (equalTo) - **verifyAllFields** (equalTo) - **verifyChecked** (locator) - **isChecked** (locator) Gets whether a toggle-button (checkbox/radio) is checked. Fails if the specified element doesn't exist or isn't a toggle-button. - **verifyCursorPosition** (locator - equalTo) - **verifyEditable** (locator) - **waitForElementEditable** (locator) - **verifyNotAllButtons** (equalTo) - **verifyNotAllFields** (equalTo) - **verifyNotChecked** (locator) - **verifyNotCursorPosition** (locator - equalTo) - **verifyNotEditable** (locator) - **verifyNotSelectedIds** (selectLocator - equalTo) - **verifyNotSelectedIndexes** (selectLocator - equalTo) - **verifyNotSelectedLabels** (selectLocator - equalTo) - **verifyNotSelectedValues** (selectLocator - equalTo) - **verifyNotSelectedOptions** (selectLocator - equalTo) - **verifyNotSomethingSelected** (selectLocator) - **verifyNotValue** (locator - equalTo) - **verifySelectedIds** (selectLocator - equalTo) - **verifySelectedIndexes** (selectLocator - equalTo) - **verifySelectedLabels** (selectLocator - equalTo) - **verifySelectedValues** (selectLocator - equalTo) - **verifySelectOptions** (selectLocator - equalTo) - **verifySomethingSelected** (selectLocator) - **isSomethingSelected** (selectLocator) Determines whether some option in a drop-down menu is selected. - **verifyValue** (locator - equalTo) - **assertAllButtons** (equalTo) - **assertAllFields** (equalTo) - **assertChecked** (locator) - **assertCursorPosition** (locator - equalTo) - **assertEditable** (locator) - **assertNotAllButtons** - **assertNotAllFields** - **assertNotChecked** (locator) - **assertNotCursorPosition** (locator - equalTo) - **assertNotEditable** (locator) - **assertNotSelectedIds** (selectLocator - equalTo) - **assertNotSelectedIndexes** (selectLocator - equalTo) - **assertNotSelectedLabels** (selectLocator - equalTo) - **assertNotSelectedValues** (selectLocator - equalTo) - **assertNotSelectOptions** (selectLocator - equalTo) - **assertNotSomethingSelected** (selectLocator) - **assertNotValue** (locator - equalTo) - **assertSelectedIds** (selectLocator - equalTo) - **assertSelectedIndexes** (selectLocator - equalTo) - **assertSelectedLabels** (selectLocator - equalTo) - **assertSelectedValues** (selectLocator - equalTo) - **assertSelectOptions** (selectLocator - equalTo) - **assertSomethingSelected** (selectLocator) - **assertValue** (locator - equalTo) - **waitForAllWindowIds** (equalTo) - **waitForAllWindowNames** (equalTo) - **waitForAllWindowTitles** (equalTo) - **waitForAttributeFromAllWindows** (attributeName - equalTo) - **waitForLocation** (equalTo) - **waitForNotAllWindowIds** (equalTo) - **waitForNotAllWindowNames** (equalTo) - **waitForNotAllWindowTitles** (equalTo) - **waitForNotAttributeFromAllWindows** (attributeName - equalTo) - **WaitForNotLocation** (equalTo) - **waitForNotTitle** (equalTo) - **waitForTitle** (equalTo) - **storeAllButtons** - **storeAllFields** - **storeChecked** (locator - variable) - **storeCursorPosition** (locator - variable) - **storeEditable** (locator - variable) - **storeSelectedIds** (selectLocator - variable) - **storeSelectedIndexes** (selectLocator - variable) - **storeSelectedLabels** (selectLocator - variable) - **storeSelectedValues** (selectLocator - variable) - **storeSelectOptions** (selectLocator - variable) - **storeSomethingSelected** (selectLocator - variable) - **storeValue** (locator - variable) ## Keyboard Events - **keyDown** (locator - keySequence) Simulates a user pressing a key (without releasing it yet). - **keyDownAndWait** (locator - keySequence) Simulates a user pressing a key (without releasing it yet). - **keyPress** (locator - keySequence) Simulates a user pressing and releasing a key. - **keyPressAndWait** (locator - keySequence) Simulates a user pressing and releasing a key. - **keyUp** (locator - keySequence) Simulates a user releasing a key. - **keyUpAndWait** (locator - keySequence) Simulates a user releasing a key. - **type** (locator - value) Sets the value of an input field, as though you typed it in. Can also be used to set the value of combo boxes, check boxes, etc. In these cases, value should be the value of the option selected, not the visible text. - **typeAndWait** (locator - value) Sets the value of an input field, as though you typed it in. Can also be used to set the value of combo boxes, check boxes, etc. In these cases, value should be the value of the option selected, not the visible text. - **typeKeys** (locator - value) Simulates keystroke events on the specified element, as though you typed the value key-by-key. This is a convenience method for calling keyDown, keyUp, keyPress for every character in the specified string; this is useful for dynamic UI widgets (like auto-completing combo boxes) that require explicit key events. Unlike the simple "type" command, which forces the specified value into the page directly, this command may or may not have any visible effect, even in cases where typing keys would normally have a visible effect. For example, if you use "typeKeys" on a form element, you may or may not see the results of what you typed in the field. In some cases, you may need to use the simple "type" command to set the value of the field and then the "typeKeys" command to send the keystroke events corresponding to what you just typed. - **sendKeys** (locator - value) - **typeKeysAndWait** (locator - value) Simulates keystroke events on the specified element, as though you typed the value key-by-key. This is a convenience method for calling keyDown, keyUp, keyPress for every character in the specified string; this is useful for dynamic UI widgets (like auto-completing combo boxes) that require explicit key events. Unlike the simple "type" command, which forces the specified value into the page directly, this command may or may not have any visible effect, even in cases where typing keys would normally have a visible effect. For example, if you use "typeKeys" on a form element, you may or may not see the results of what you typed in the field. In some cases, you may need to use the simple "type" command to set the value of the field and then the "typeKeys" command to send the keystroke events corresponding to what you just typed. ## Keyboard Modifiers - **altKeyDown** Press the alt key and hold it down until doAltUp() is called or a new page is loaded. - **altKeyDownAndWait** Press the alt key and hold it down until doAltUp() is called or a new page is loaded. - **altKeyUp** Release the alt key. - **altKeyUpAndWait** Release the alt key. - **controlKeyDown** Press the control key and hold it down until doControlUp() is called or a new page is loaded. - **controlKeyDownAndWait** Press the control key and hold it down until doControlUp() is called or a new page is loaded. - **controlKeyUp** Release the control key. - **controlKeyUpAndWait** Release the control key. - **metaKeyDown** Press the meta key and hold it down until doMetaUp() is called or a new page is loaded. - **metaKeyUp** Release the meta key. - **metaKeyUpAndWait** Release the meta key. - **shiftKeyDown** Press the shift key and hold it down until doShiftUp() is called or a new page is loaded. - **shiftKeyDownAndWait** Press the shift key and hold it down until doShiftUp() is called or a new page is loaded. - **shiftKeyUp** Release the shift key. - **shiftKeyUpAndWait** Release the shift key. ## Mouse Events - **mouseDown** (locator) Simulates a user pressing the left mouse button (without releasing it yet) on the specified element. - **mouseDownAndWait** (locator) Simulates a user pressing the left mouse button (without releasing it yet) on the specified element. - **mouseDownAt** (locator - coordString) Simulates a user pressing the left mouse button (without releasing it yet) at the specified location. - **mouseDownAtAndWait** (locator - coordString) Simulates a user pressing the left mouse button (without releasing it yet) at the specified location. - **mouseDownRight** (locator) Simulates a user pressing the right mouse button (without releasing it yet) on the specified element. - **mouseDownRightAndWait** (locator) Simulates a user pressing the right mouse button (without releasing it yet) on the specified element. - **mouseDownRightAt** (locator - coordString) Simulates a user pressing the right mouse button (without releasing it yet) at the specified location. - **mouseDownRightAtAndWait** (locator - coordString) Simulates a user pressing the right mouse button (without releasing it yet) at the specified location. - **mouseMove** (locator) Simulates a user pressing the mouse button (without releasing it yet) on the specified element. - **mouseMoveAndWait** (locator) Simulates a user pressing the mouse button (without releasing it yet) on the specified element. - **mouseMoveAt** (locator - coordString) Simulates a user pressing the mouse button (without releasing it yet) on the specified element. - **mouseMoveAtAndWait** (locator - coordString) Simulates a user pressing the mouse button (without releasing it yet) on the specified element. - **mouseOut** (locator) Simulates a user moving the mouse pointer away from the specified element. - **mouseOutAndWait** (locator) Simulates a user moving the mouse pointer away from the specified element. - **mouseOver** (locator) Simulates a user hovering a mouse over the specified element. - **mouseOverAndWait** (locator) Simulates a user hovering a mouse over the specified element. - **mouseUp** (locator) Simulates the event that occurs when the user releases the mouse button (i.e., stops holding the button down) on the specified element. - **mouseUpAndWait** (locator) Simulates the event that occurs when the user releases the mouse button (i.e., stops holding the button down) on the specified element. - **mouseUpAt** (locator - coordString) Simulates the event that occurs when the user releases the mouse button (i.e., stops holding the button down) at the specified location. - **mouseUpAtAndWait** (locator - coordString) Simulates the event that occurs when the user releases the mouse button (i.e., stops holding the button down) at the specified location. - **mouseUpRight** (locator) Simulates the event that occurs when the user releases the right mouse button (i.e., stops holding the button down) on the specified element. - **mouseUpRightAndWait** (locator) Simulates the event that occurs when the user releases the right mouse button (i.e., stops holding the button down) on the specified element. - **mouseUpRightAt** (locator - coordString) Simulates the event that occurs when the user releases the right mouse button (i.e., stops holding the button down) at the specified location. - **mouseUpRightAtAndWait** (locator - coordString) Simulates the event that occurs when the user releases the right mouse button (i.e., stops holding the button down) at the specified location. ## Popups and Menus - **answerOnNextPrompt** (answer) Instructs Selenium to return the specified answer string in response to the next JavaScript prompt [window.prompt()]. - **answerOnNextPromptAndWait** (answer) Instructs Selenium to return the specified answer string in response to the next JavaScript prompt [window.prompt()]. - **chooseCancelOnNextConfirmation** By default, Selenium's overridden window.confirm() function will return true, as if the user had manually clicked OK; after running this command, the next call to confirm() will return false, as if the user had clicked Cancel. Selenium will then resume using the default behavior for future confirmations, automatically returning true (OK) unless/until you explicitly call this command for each confirmation. Take note - every time a confirmation comes up, you must consume it with a corresponding getConfirmation, or else the next selenium operation will fail. - **chooseCancelOnNextConfirmationAndWait** By default, Selenium's overridden window.confirm() function will return true, as if the user had manually clicked OK; after running this command, the next call to confirm() will return false, as if the user had clicked Cancel. Selenium will then resume using the default behavior for future confirmations, automatically returning true (OK) unless/until you explicitly call this command for each confirmation. Take note - every time a confirmation comes up, you must consume it with a corresponding getConfirmation, or else the next selenium operation will fail. - **chooseOkOnNextConfirmation** Undo the effect of calling chooseCancelOnNextConfirmation. Note that Selenium's overridden window.confirm() function will normally automatically return true, as if the user had manually clicked OK, so you shouldn't need to use this command unless for some reason you need to change your mind prior to the next confirmation. After any confirmation, Selenium will resume using the default behavior for future confirmations, automatically returning true (OK) unless/until you explicitly call chooseCancelOnNextConfirmation for each confirmation. Take note - every time a confirmation comes up, you must consume it with a corresponding getConfirmation, or else the next selenium operation will fail. - **chooseOkOnNextConfirmationAndWait** Undo the effect of calling chooseCancelOnNextConfirmation. Note that Selenium's overridden window.confirm() function will normally automatically return true, as if the user had manually clicked OK, so you shouldn't need to use this command unless for some reason you need to change your mind prior to the next confirmation. After any confirmation, Selenium will resume using the default behavior for future confirmations, automatically returning true (OK) unless/until you explicitly call chooseCancelOnNextConfirmation for each confirmation. Take note - every time a confirmation comes up, you must consume it with a corresponding getConfirmation, or else the next selenium operation will fail. - **contextMenu** (locator) Simulates opening the context menu for the specified element (as might happen if the user "right-clicked" on the element). - **contextMenuAndWait** (locator) Simulates opening the context menu for the specified element (as might happen if the user "right-clicked" on the element). - **contextMenuAt** (locator - coordString) Simulates opening the context menu for the specified element (as might happen if the user "right-clicked" on the element). - **contextMenuAtAndWait** (locator - coordString) Simulates opening the context menu for the specified element (as might happen if the user "right-clicked" on the element). - **selectPopUp** (windowID) Simplifies the process of selecting a popup window (and does not offer functionality beyond what selectWindow() already provides). If windowID is either not specified, or specified as "null", the first non-top window is selected. The top window is the one that would be selected by selectWindow() without providing a windowID . This should not be used when more than one popup window is in play. Otherwise, the window will be looked up considering windowID as the following in order: 1) the "name" of the window, as specified to window.open(); 2) a javascript variable which is a reference to a window; and 3) the title of the window. This is the same ordered lookup performed by selectWindow . - **deselectPopUp** Selects the main window. Functionally equivalent to using selectWindow() and specifying no value for windowID. - **getAlert** Retrieves the message of a JavaScript alert generated during the previous action, or fail if there were no alerts. Getting an alert has the same effect as manually clicking OK. If an alert is generated but you do not consume it with getAlert, the next Selenium action will fail. Under Selenium, JavaScript alerts will NOT pop up a visible alert dialog. Selenium does NOT support JavaScript alerts that are generated in a page's onload() event handler. In this case a visible dialog WILL be generated and Selenium will hang until someone manually clicks OK. - **getConfirmation** Retrieves the message of a JavaScript confirmation dialog generated during the previous action. By default, the confirm function will return true, having the same effect as manually clicking OK. This can be changed by prior execution of the chooseCancelOnNextConfirmation command. If an confirmation is generated but you do not consume it with getConfirmation, the next Selenium action will fail. NOTE: under Selenium, JavaScript confirmations will NOT pop up a visible dialog. NOTE: Selenium does NOT support JavaScript confirmations that are generated in a page's onload() event handler. In this case a visible dialog WILL be generated and Selenium will hang until you manually click OK. - **getPrompt** Retrieves the message of a JavaScript question prompt dialog generated during the previous action. Successful handling of the prompt requires prior execution of the answerOnNextPrompt command. If a prompt is generated but you do not get/verify it, the next Selenium action will fail. NOTE: under Selenium, JavaScript prompts will NOT pop up a visible dialog. NOTE: Selenium does NOT support JavaScript prompts that are generated in a page's onload() event handler. In this case a visible dialog WILL be generated and Selenium will hang until someone manually clicks OK. - **getWhetherThisFrameMatchFrameExpression** (currentFrameString - target) Determine whether current/locator identify the frame containing this running code. This is useful in proxy injection mode, where this code runs in every browser frame and window, and sometimes the selenium server needs to identify the "current" frame. In this case, when the test calls selectFrame, this routine is called for each frame to figure out which one has been selected. The selected frame will return true, while all others will return false. - **getWhetherThisWindowMatchWindowExpression** (currentWindowString - target) Determine whether currentWindowString plus target identify the window containing this running code. This is useful in proxy injection mode, where this code runs in every browser frame and window, and sometimes the selenium server needs to identify the "current" window. In this case, when the test calls selectWindow, this routine is called for each window to figure out which one has been selected. The selected window will return true, while all others will return false. ## Browser Window - **verifyAllWindowIds** (equalTo) - **verifyAllWindowNames** (equalTo) - **verifyAllWindowTitles** (equalTo) - **verifyAttributeFromAllWindows** (attributeName - equalTo) - **verifyLocation** (equalTo) - **verifyNotAllWindowIds** (equalTo) - **verifyNotAllWindowNames** (equalTo) - **verifyNotAttributeFromAllWindows** (attributeName - equalTo) - **verifyNotLocation** (equalTo) - **verifyNotTitle** (equalTo) - **verifyTitle** (equalTo) - **assertAllWindowIds** (equalTo) - **assertAllWindowNames** (equalTo) - **assertAllWindowTitles** (equalTo) - **assertAttributeFromAllWindows** (attributeName - equalTo) - **assertLocation** (equalTo) - **assertNotAllWindowIds** (equalTo) - **assertNotAllWindowNames** (equalTo) - **assertNotAllWindowTitles** (equalTo) - **assertNotAttributeFromAllWindows** (attributeName) - **assertNotLocation** (equalTo) - **assertNotTitle** (equalTo) - **assertTitle** (equalTo) - **assert** Check that a variable is an expected value. The variable's value will be converted to a string for comparison. The test will stop if the assert fails. - **validateXml** Custom command by TestingBot. Validates if the document contents is valid XML. - **waitForAlert** (equalTo) - **waitForAlertNotPresent** - **waitForAlertPresent** - **waitForConfirmation** (equalTo) - **waitForConfirmationNotPresent** - **waitForConfirmationPresent** - **waitForNotAlert** (equalTo) - **waitForNotConfirmation** (equalTo) - **waitForNotPrompt** (equalTo) - **waitForPrompt** (equalTo) - **waitForPromptNotPresent** - **waitForPromptPresent** - **storeAllWindowIds** - **storeAllWindowNames** - **storeAllWindowTitles** - **storeAttributeFromAllWindows** (attributeName - variable) - **storeLocation** - **storeTitle** - **storeAllWindowHandles** - **storeNewWindowHandle** - **storeWindowHandle** - **setWindowSize** (size) Sets the current window size. ## Cookies - **verifyCookie** (equalTo) - **verifyCookieByName** (name - equalTo) - **verifyCookieNotPresent** (name) - **verifyCookiePresent** (name) - **verifyNotCookie** (equalTo) - **verifyNotCookieByName** (name - equalTo) - **assertCookie** (equalTo) - **assertCookieByName** (name - equalTo) - **assertCookieNotPresent** (name) - **assertCookiePresent** (name) - **assertNotCookie** (equalTo) - **assertNotCookieByName** (name - equalTo) - **createCookie** (nameValuePair - optionsString) Create a new cookie whose path and domain are same with those of current page under test, unless you specified a path for this cookie explicitly. - **deleteAllVisibleCookies** Calls deleteCookie with recurse=true on all cookies visible to the current page. As noted on the documentation for deleteCookie, recurse=true can be much slower than simply deleting the cookies using a known domain/path. - **deleteCookie** (name - optionsString) Delete a named cookie with specified path and domain. Be careful; to delete a cookie, you need to delete it using the exact same path and domain that were used to create the cookie. If the path is wrong, or the domain is wrong, the cookie simply won't be deleted. Also note that specifying a domain that isn't a subset of the current domain will usually fail. Since there's no way to discover at runtime the original path and domain of a given cookie, we've added an option called 'recurse' to try all sub-domains of the current domain with all paths that are a subset of the current path. Beware; this option can be slow. In big-O notation, it operates in O(n\*m) time, where n is the number of dots in the domain name and m is the number of slashes in the path. - **storeCookie** - **storeCookieByName** (name - variable) - **storeCookiePresent** (name - variable) ## Page Content - **verifyAllLinks** (equalTo) - **verifyAttribute** (attributeLocator - equalTo) - **verifyBodyText** (equalTo) - **verifyElementNotPresent** (locator) - **verifyElementPresent** (locator) - **verifyHtmlSource** (equalTo) - **verifyNotAllLinks** (equalTo) - **verifyNotAttribute** (attributeLocator) - **verifyNotBodyText** (equalTo) - **verifyNotHtmlSource** (equalTo) - **verifyNotOrdered** (locator1 - locator2) - **verifyNotTable** (tableCellAddress - equalTo) - **verifyNotText** (locator - equalTo) - **verifyNotVisible** (locator) - **verifyOrdered** (locator1 - locator2) - **verifyTable** (tableCellAddress - equalTo) - **verifyText** (locator - equalTo) - **verifyTextNotPresent** (pattern) - **verifyTextPresent** (pattern) - **verifyVisible** (locator) - **assertAllLinks** (equalTo) - **assertAttribute** (attributeLocator - equalTo) - **assertBodyText** (equalTo) - **assertElementNotPresent** (locator) - **assertElementPresent** (locator) - **assertHtmlSource** (equalTo) - **assertNotAllLinks** (equalTo) - **assertNotAttribute** (attributeLocator - equalTo) - **assertNotBodyText** (equalTo) - **assertNotHtmlSource** (equalTo) - **assertNotOrdered** (locator1 - locator2) - **assertNotTable** (tableCellAddress - equalTo) - **assertNotText** (locator - equalTo) - **assertNotVisible** (locator) - **assertOrdered** (locator1 - locator2) - **assertTable** (tableCellAddress - equalTo) - **assertText** (locator - equalTo) - **assertTextNotPresent** (pattern) - **assertTextPresent** (pattern) - **assertVisible** (locator) - **waitForAllLinks** (equalTo) - **waitForAttribute** (attributeLocator - equalTo) - **waitForBodyText** (equalTo) - **waitForElementPresent** (locator) - **waitForHtmlSource** (equalTo) - **waitForNotAllLinks** (equalTo) - **waitForNotAttribute** (attributeLocator - equalTo) - **waitForNotBodyText** (equalTo) - **waitForElementNotPresent** (locator) - **waitForElementNotVisible** (locator) - **waitForElementVisible** (locator) - **waitForNotHtmlSource** (equalTo) - **waitForNotOrdered** (locator1 - locator2) - **waitForNotTable** (tableCellAddress - equalTo) - **waitForNotText** (locator - equalTo) - **waitForNotVisible** (locator) - **waitForOrdered** (locator1 - locator2) - **waitForTable** (tableCellAddress - equalTo) - **waitForText** (locator - equalTo) - **waitForTextNotPresent** (pattern) - **waitForTextPresent** (pattern) - **waitForVisible** (locator) - **storeAllLinks** - **storeAttribute** (attributeLocator - variable) - **storeBodyText** - **storeElementPresent** (locator - variable) - **storeHtmlSource** - **storeTable** (tableCellAddress - variable) - **storeText** (locator - variable) - **storeTextPresent** (pattern - variable) - **storeVisible** (locator - variable) ## Page Positioning - **verifyElementHeight** (locator - equalTo) - **verifyElementIndex** (locator - equalTo) - **verifyElementPositionLeft** (locator - equalTo) - **verifyElementPositionTop** (locator - equalTo) - **verifyElementWidth** (locator - equalTo) - **verifyNotElementHeight** (locator - equalTo) - **verifyNotElementIndex** (locator - equalTo) - **verifyNotElementPositionLeft** (locator - equalTo) - **verifyNotElementPositionTop** (locator - equalTo) - **verifyNotElementWidth** (locator - equalTo) - **assertElementHeight** (locator - equalTo) - **assertElementIndex** (locator - equalTo) - **assertElementPositionLeft** (locator - equalTo) - **assertElementPositionTop** (locator - equalTo) - **assertElementWidth** (locator - equalTo) - **assertNotElementHeight** (locator - equalTo) - **assertNotElementIndex** (locator - equalTo) - **assertNotElementPositionLeft** (locator - equalTo) - **assertNotElementPositionTop** (locator - equalTo) - **assertNotElementWidth** - **storeElementHeight** (locator - variable) - **storeElementIndex** (locator - variable) - **storeElementPositionLeft** (locator - variable) - **storeElementPositionTop** (locator - variable) - **storeElementWidth** (locator - variable) ## Popups - **verifyAlert** (equalTo) - **verifyAlertNotPresent** - **verifyAlertPresent** - **isAlertPresent** Has an alert occurred? This function never throws an exception - **verifyConfirmation** (equalTo) - **verifyConfirmationNotPresent** - **verifyConfirmationPresent** - **isConfirmationPresent** Has confirm() been called? This function never throws an exception - **verifyNotAlert** (equalTo) - **verifyNotConfirmation** (equalTo) - **verifyNotPrompt** (equalTo) - **verifyPrompt** (equalTo) - **verifyPromptNotPresent** - **verifyPromptPresent** - **isPromptPresent** Has a prompt occurred? This function never throws an exception - **assertAlert** (equalTo) - **assertAlertNotPresent** - **assertAlertPresent** - **assertConfirmation** (equalTo) - **assertConfirmationNotPresent** - **assertConfirmationPresent** - **assertNotAlert** (equalTo) - **assertNotConfirmation** (equalTo) - **assertNotPrompt** (equalTo) - **assertPrompt** (equalTo) - **assertPromptNotPresent** - **assertPromptPresent** - **waitForElementHeight** (locator - equalTo) - **waitForElementIndex** (locator - equalTo) - **waitForElementPositionLeft** (locator - equalTo) - **waitForElementPositionTop** (locator - equalTo) - **waitForElementWidth** (locator - equalTo) - **waitForNotElementHeight** (locator - equalTo) - **waitForNotElementIndex** (locator - equalTo) - **waitForNotElementPositionLeft** (locator - equalTo) - **waitForNotElementPositionTop** (locator - equalTo) - **waitForNotElementWidth** (locator - equalTo) - **waitForPopUp** (windowID - timeout) Waits for a popup window to appear and load up. - **storeAlert** - **storeAlertPresent** - **storeConfirmation** - **storeConfirmationPresent** - **storePrompt** - **storePromptPresent** ## Selenium - **verifyEval** (script - equalTo) - **verifyExpression** (expression - equalTo) - **verifyMouseSpeed** - **verifyNotEval** (script - equalTo) - **verifyNotExpression** (expression - equalTo) - **verifyNotMouseSpeed** - **verifyNotSpeed** - **verifyNotXpathCount** (xpath - equalTo) - **verifySpeed** - **verifyXpathCount** (xpath - equalTo) - **assertEval** (script - equalTo) - **assertExpression** (expression - equalTo) - **assertMouseSpeed** (equalTo) - **assertNotEval** (script - equalTo) - **assertNotExpression** (expression - equalTo) - **assertNotMouseSpeed** (equalTo) - **assertNotSpeed** (equalTo) - **assertNotXpathCount** (xpath - equalTo) - **assertSpeed** (equalTo) - **assertXpathCount** (xpath - equalTo) - **waitForAllButtons** (equalTo) - **waitForAllFields** (equalTo) - **waitForChecked** (locator) - **waitForCursorPosition** (locator - equalTo) - **waitForCondition** (script - timeout) Runs the specified JavaScript snippet repeatedly until it evaluates to "true". The snippet may have multiple lines, but only the result of the last line will be considered. Note that, by default, the snippet will be run in the runner's test window, not in the window of your application. To get the window of your application, you can use the JavaScript snippet selenium.browserbot.getCurrentWindow(), and then run your JavaScript in there - **waitForEditable** (locator) - **waitForEval** (script - equalTo) - **waitForExpression** (expression - equalTo) - **waitForMouseSpeed** (equalTo) - **waitForNotAllButtons** (equalTo) - **waitForNotAllFields** (equalTo) - **waitForNotChecked** (locator) - **waitForNotCursorPosition** (locator - equalTo) - **waitForNotEditable** (locator) - **waitForNotEval** (script - equalTo) - **waitForNotExpression** (expression - equalTo) - **waitForNotMouseSpeed** (equalTo) - **waitForNotSelectedIds** (selectLocator - equalTo) - **waitForNotSelectedIndexes** (selectLocator - equalTo) - **waitForNotSelectedLabels** (selectLocator - equalTo) - **waitForNotSelectedValues** (selectLocator - equalTo) - **waitForNotSelectOptions** (selectLocator - equalTo) - **waitForNotSomethingSelected** (selectLocator) - **waitForNotSpeed** (equalTo) - **waitForNotValue** (locator - equalTo) - **waitForNotXpathCount** (xpath - equalTo) - **waitForSelectedIds** (selectLocator - equalTo) - **waitForSelectedIndexes** (selectLocator - equalTo) - **waitForSelectedLabels** (selectLocator - equalTo) - **waitForSelectedValues** (selectLocator - equalTo) - **waitForSelectOptions** (selectLocator - equalTo) - **waitForSomethingSelected** (selectLocator) - **waitForSpeed** (equalTo) - **waitForValue** (locator - equalTo) - **waitForXpathCount** (xpath - equalTo) - **storeEval** (script - variable) - **storeExpression** (expression - variable) - **store** (expression - variable) This command is a synonym for storeExpression. - **storeMouseSpeed** - **storeSpeed** - **storeXpathCount** (xpath - variable) ## Screenshots - **captureEntirePageScreenshot** (filename - kwargs) Saves the entire contents of the current window canvas to a PNG file. Contrast this with the captureScreenshot command, which captures the contents of the OS viewport (i.e. whatever is currently being displayed on the monitor), and is implemented in the RC only. Currently this only works in Firefox when running in chrome mode, and in IE non-HTA using the EXPERIMENTAL "Snapsie" utility. The Firefox implementation is mostly borrowed from the Screengrab! Firefox extension. Please see http://www.screengrab.org and http://snapsie.sourceforge.net/ for details. ## Selenium Settings - **waitForCookie** (equalTo) - **waitForCookieByName** (name - equalTo) - **waitForCookieNotPresent** (name) - **waitForCookiePresent** (name) - **waitForNotCookie** (equalTo) - **waitForNotCookieByName** (name - equalTo) - **addLocationStrategy** (strategyName - functionDefinition) Defines a new function for Selenium to locate elements on the page. For example, if you define the strategy "foo", and someone runs click("foo=blah"), we'll run your function, passing you the string "blah", and click on the element that your function returns, or throw an "Element not found" error if your function returns null. We'll pass three arguments to your function: locator: the string the user passed in inWindow: the currently selected window inDocument: the currently selected document The function must return null if the element can't be found. - **allowNativeXpath** (allow) Specifies whether Selenium should use the native in-browser implementation of XPath (if any native version is available); if you pass "false" to this function, we will always use our pure-JavaScript xpath library. Using the pure-JS xpath library can improve the consistency of xpath element locators between different browser vendors, but the pure-JS version is much slower than the native implementations. - **ignoreAttributesWithoutValue** (ignore) Specifies whether Selenium will ignore xpath attributes that have no value, i.e. are the empty string, when using the non-native xpath evaluation engine. You'd want to do this for performance reasons in IE. However, this could break certain xpaths, for example an xpath that looks for an attribute whose value is NOT the empty string. The hope is that such xpaths are relatively rare, but the user should have the option of using them. Note that this only influences xpath evaluation when using the ajaxslt engine (i.e. not "javascript-xpath"). - **setBrowserLogLevel** (logLevel) Sets the threshold for browser-side logging messages; log messages beneath this threshold will be discarded. Valid logLevel strings are: "debug", "info", "warn", "error" or "off". To see the browser logs, you need to either show the log window in GUI mode, or enable browser-side logging in Selenium RC. - **setMouseSpeed** (pixels) Configure the number of pixels between "mousemove" events during dragAndDrop commands (default=10). Setting this value to 0 means that we'll send a "mousemove" event to every single pixel in between the start location and the end location; that can be very slow, and may cause some browsers to force the JavaScript to timeout. If the mouse speed is greater than the distance between the two dragged objects, we'll just send one "mousemove" at the start location and then one final one at the end location. - **setSpeed** (value) Set execution speed (i.e., set the millisecond length of a delay which will follow each selenium operation). By default, there is no such delay, i.e., the delay is 0 milliseconds. - **getSpeed** Get execution speed (i.e., get the millisecond length of the delay following each selenium operation). By default, there is no such delay, i.e., the delay is 0 milliseconds. See also setSpeed. - **setTimeout** (timeout) Specifies the amount of time that Selenium will wait for actions to complete. Actions that require waiting include "open" and the "waitFor\*" actions. The default timeout is 30 seconds. - **useXpathLibrary** (libraryName) Allows choice of one of the available libraries. ## Special - **addScript** (scriptContent - scriptTagId) Loads script content into a new script tag in the Selenium document. This differs from the runScript command in that runScript adds the script tag to the document of the AUT, not the Selenium document. The following entities in the script content are replaced by the characters they represent: \< \> & The corresponding remove command is removeScript. - **assignId** (locator - identifier) Temporarily sets the "id" attribute of the specified element, so you can locate it in the future using its ID rather than a slow/complicated XPath. This ID will disappear once the page is reloaded. - **echo** (message) Prints the specified message into the third table cell in your Selenese tables. Useful for debugging. - **fireEvent** (locator - eventName) Explicitly simulate an event, to trigger the corresponding "onevent" handler. - **highlight** (locator) Briefly changes the backgroundColor of the specified element yellow. Useful for debugging. - **rollup** (rollupName - kwargs) Executes a command rollup, which is a series of commands with a unique name, and optionally arguments that control the generation of the set of commands. If any one of the rolled-up commands fails, the rollup is considered to have failed. Rollups may also contain nested rollups. - **runScript** (script) Creates a new "script" tag in the body of the current test window, and adds the specified text into the body of the command. Scripts run in this way can often be debugged more easily than scripts executed using Selenium's "getEval" command. Beware that JS exceptions thrown in these script tags aren't managed by Selenium, so you should probably wrap your script in try/catch blocks if there is any chance that the script will throw an exception. - **getEval** (command) Gets the result of evaluating the specified JavaScript snippet. The snippet may have multiple lines, but only the result of the last line will be returned. Note that, by default, the snippet will run in the context of the "selenium" object itself, so this will refer to the Selenium object. Use window to refer to the window of your application, e.g. window.document.getElementById('foo') If you need to use a locator to refer to a single element in your application page, you can use this.browserbot.findElement("id=foo") where "id=foo" is your locator. - **getLocation** Gets the absolute URL of the current page. - **getTitle** Gets the title of the current page. - **getBodyText** Gets the entire text of the page. - **getValue** (locator) Gets the (whitespace-trimmed) value of an input field (or anything else with a value parameter). For checkbox/radio elements, the value will be "on" or "off" depending on whether the element is checked or not. - **getText** (locator) Gets the text of an element. This works for any element that contains text. This command uses either the textContent (Mozilla-like browsers) or the innerText (IE-like browsers) of the element, which is the rendered text shown to the user. - **getTable** (tableCellAddress) Gets the text from a cell of a table. The cellAddress syntax tableLocator.row.column, where row and column start at 0. - **pause** (time) Wait for the specified amount of time (in milliseconds) - **comment** (text) Custom command by TestingBot. Add a comment between your test steps. This will show up in the test results. - **getHtmlSource** Returns the entire HTML source between the opening and closing "html" tags. - **stopTest** (condition) Custom command by TestingBot. Will stop the test if the supplied condition is met. Other steps, which resulted in an error, after a successful stopTest, will no longer indicate a test as failed. ## Common - **waitForPageToLoad** (timeout) Waits for a new page to load. You can use this command instead of the "AndWait" suffixes, "clickAndWait", "selectAndWait", "typeAndWait" etc. (which are only available in the JS API). Selenium constantly keeps track of new pages loading, and sets a "newPageLoaded" flag when it first notices a page load. Running any other Selenium command after turns the flag to false. Hence, if you want to wait for a page to load, you must wait immediately after a Selenium command that caused a page-load. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/codeless-automation/faq # Selenium IDE FAQ TestingBot supports importing and running Selenium IDE tests on all our browsers and devices. Please find some questions and answers below. ## FAQ - **How do I use `selenium-side-runner` with TestingBot?** You can easily run your tests with `selenium-side-runner`, try this command for example ```bash selenium-side-runner --server http://key:secret@hub.testingbot.com/wd/hub -c "browserName='internet explorer' version='11.0' platform='Windows 8.1'" my-test-file.side ``` This will run the test on our Grid on IE11 - the test results will appear in the TestingBot dashboard. - **Do you support the old Selenium IDE format? What about the .SIDE format?** We support both versions in our TestLab. You can upload the old format (in `HTML` format) or newly generated tests (in `SIDE` format). - **Do you support control flow?** We do not yet support control flow, like `if`, `else`, `while`, ... If you need this feature, please [contact us](https://testingbot.com/contact/new). - **Do you support mobile testing with Codeless Automation?** We support running your tests on our iOS and Android simulators/emulators. Please consider using `css selectors` as much as possible, as Appium (which we use to control iOS/Android) does not support XPath selectors. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/codeless-automation/variables # Storing Selenium IDE Variables With Selenium IDE it is possible to `store` variables. These variables can then be used in later commands during the same test flow. ## How to store a variable To store a variable during a test, please use the `store` command: ```bash store - javascript(return "test" + Math.floor(Math.random()*11111) + "@test.com") - emailAddress ``` This example will store a variable called `emailAddress` and its value will be a random `"test....@test.com"` string. Storing a variable happens in the context of the page. You can use any kind of javascript and reference DOM elements or other functions available on the page. The return value is stored. ## How to use a variable Once you have stored a variable, you can use it in subsequent steps like this: `${emailAddress}` ## Examples Below are some examples on how to [store and use variables](https://www.selenium.dev/selenium-ide/docs/en/api/commands#store). ```bash storeText("css=.container:nth-child(1) > span:nth-child(3)","myVar") ``` This will store the text from element with CSS selector `.container:nth-child(1) > span:nth-child(3)` in a variable called `myVar`. ```bash type('id=inputField','${myVar}') ``` This will type the text stored in the `myVar` variable to an element with `ID=inputField` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/ai # AI Testing TestingBot provides AI solutions for QA and developers to automate their tests and other tasks. Here are some of the AI solutions we are providing: - [TestingBot AI Insights](https://testingbot.com/support/ai/insights) Get an AI explanation of why a test failed: root cause, timeline, evidence and suggested fixes. Works with Selenium, Appium, Playwright, Espresso, XCUITest and Maestro. - [TestingBot AI Chat](https://testingbot.com/support/ai/chat) Chat with an AI operator and instruct a remote browser. Generate automated tests from your instructions. - [Codeless Tests](https://testingbot.com/support/ai/codeless) Instruct our AI agent what to test. The AI agent will run tests against your website on regular intervals and alert if a test fails. - [TestingBot MCP Server](https://testingbot.com/support/ai/mcp) Connect TestingBot with AI models, using the TestingBot MCP Server. - [Testing WebMCP Tools](https://testingbot.com/support/ai/webmcp) Test your website's WebMCP tools (navigator.modelContext) on real browsers with Selenium, Playwright and Puppeteer. - [Other AI integrations such as Browser-Use, Stagehand and Magnitude](https://testingbot.com/support/ai/integrations) ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/ai/insights # TestingBot AI Insights TestingBot AI Insights is an AI-powered test failure analysis feature. When an automated test fails, whether it is a Selenium, Appium, Playwright, Puppeteer, Cypress, Espresso, XCUITest or Maestro test, AI Insights reads the test's logs and explains, in plain language, why it failed: the most likely root cause, a timeline of what happened, the supporting evidence from the logs, and a suggested fix, together with a confidence score. Instead of scrolling through a long command log to work out what went wrong, you open the test and read a short, structured explanation. ![TestingBot AI Insights tab showing a failed test's root cause, confidence score, timeline, evidence and suggested fixes](https://testingbot.com/assets/support/ai/insights-20af55cea64b10a0eb94765d2e0cdc3f7900aef9eee3853591fad7533459b5d2.svg) _The AI Analysis tab explains a failed test: summary, root cause, likely owner, confidence, timeline, evidence and suggested fixes._ ## How to enable it AI Insights is off by default and must be enabled per account by the account owner. Because your test logs are sent to a third-party AI provider for analysis, enabling it records an explicit consent. 1. Sign in as the **account owner** and go to [Account Settings](https://testingbot.com/members/user/edit#ai-analysis). 2. In the **AI Analysis** section, read what is sent, then tick the consent checkbox and save. 3. The **AI Analysis** tab will now appear on your test detail pages. Team members cannot enable the feature themselves: it is an account-level setting controlled by the owner. You can disable it again at any time from the same screen, which immediately stops any further data being sent. ## Using AI Insights Open a test at **Tests → (your test)** and select the **AI Analysis** tab. - **Failed tests** are analyzed automatically the first time you open the tab. The explanation streams in section by section. - **Passed tests** are not analyzed automatically, but you can choose **Analyze anyway**. - Results are **cached** , so reopening a test shows the previous analysis instantly with no extra cost. - Use **Re-analyze** to run a fresh analysis, for example after you have changed the test. ## What you get A completed analysis is presented as a set of sections: - **Summary** : one or two sentences on what the test was doing and how it failed. - **Root cause** : the single most likely reason for the failure. - **Likely owner** : where the problem most likely sits, classified as _application_, _test script_, _environment_, _flaky_ or _unknown_. - **Confidence** : a score from 0 to 100 (shown as low, medium or high). When confidence is low, the likely owner is shown as a tentative "best guess" rather than a firm verdict. - **Timeline** : the key events leading up to the failure. - **Evidence** : the specific log lines and commands that support the conclusion. - **Suggested fixes** : one to three concrete, ranked suggestions. AI Insights is an assistant, not an oracle. It can be wrong, especially on ambiguous failures such as flaky timing issues or visual differences, where it will tell you the confidence is low. Always verify before acting on a suggestion. ## What data is sent AI Insights sends a small, curated, text-only slice of the test to the AI provider. It is built from artifacts you already have on TestingBot: - The failing step and the steps around it. - A filtered, compressed tail of the most relevant driver or device log (for example the Selenium, Appium, Playwright, logcat or iOS log), focused on errors, warnings and the failure window. - For native runners (Espresso, XCUITest, Maestro), the failing test results and their stacktraces or error messages. - For Maestro, the flow definition (the YAML that lists the intended steps), which helps explain which step failed and why. - The test's status, status message and termination reason. - The environment (browser or device, version and OS). - For codeless AI tests, the natural-language test intent. ## What gets stripped Before anything leaves our servers, the curated text passes through an automated masking step that redacts detectable secrets and personal data and replaces them with placeholders such as ``: - API keys and tokens (including JWTs, AWS, Stripe, GitHub, Google and similar key formats). - Passwords, including values typed into password fields during the test. - `Authorization`, `Cookie` and similar sensitive HTTP headers. - Private keys and other high-entropy secrets. - Email addresses and card-number-shaped values. This is best-effort masking (pseudonymization), not anonymization. It significantly reduces what is shared, but it cannot guarantee that every piece of sensitive data is removed. Only enable AI Insights for tests you are authorized to share, and prefer keeping secrets out of test logs in the first place. See our guidance on [handling sensitive data](https://testingbot.com/support/web-automate/selenium/sensitive-data). ## Privacy and data handling - **AI provider** : analysis is performed by **Anthropic** (United States), our AI sub-processor, under their standard Data Processing Addendum with EU Standard Contractual Clauses. - **No training** : Anthropic does not use data sent through their commercial API to train their models. - **Short retention at the provider** : inputs and outputs are deleted by Anthropic within 30 days by default. - **Consent** : the feature is off until the account owner explicitly enables it, and it can be disabled at any time. - **Retention on TestingBot** : a generated analysis is stored with the test and is removed when the test is pruned, in line with our standard 30-day retention of test logs and assets. ## Availability and limits - There are per-account daily and monthly limits to keep usage and cost bounded. - Tests older than 30 days cannot be analyzed because their logs have been pruned. - Tests that are still running cannot be analyzed until they complete. ## Frequently Asked Questions ### Which frameworks does AI Insights support? AI Insights works with Selenium, Appium, Playwright, Puppeteer, Cypress, Espresso, XCUITest and Maestro tests, as well as codeless AI tests. For web and Appium tests it reads the command log and driver logs; for native mobile runners it reads the test results and stacktraces, the device log, and (for Maestro) the flow definition. Open the failing test or run and select the AI Analysis tab. ### Will it analyze passing tests? Not automatically. Passing tests show an "Analyze anyway" option if you want an explanation. ### How accurate is it? It is reliable for clear-cut failures such as assertion errors, element-not-found, timeouts and HTTP errors, and less certain for ambiguous cases such as flaky timing or visual differences. The confidence score reflects this, and low-confidence verdicts are clearly marked. ### How do I turn it off? The account owner can disable it from [Account Settings](https://testingbot.com/members/user/edit#ai-analysis). Disabling it immediately stops any further data being sent. ### Can I keep secrets out of the analysis entirely? Yes. We mask detectable secrets automatically, but the best practice is to avoid logging sensitive values. See [handling sensitive data](https://testingbot.com/support/web-automate/selenium/sensitive-data). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/ai/integrations # TestingBot's Remote Browser Cloud TestingBot offers a battle-tested remote browser cloud, available to run both automated tests and AI tasks. When it comes to running AI-driven browser automation tasks, selecting the right remote browser grid is crucial. **TestingBot** offers a scalable, feature-rich alternative to platforms like Browserbase, Steel.dev, Browserless and Lightpanda. At TestingBot we've been operating a browser grid since 2012, providing unmatched reliability, global coverage and enterprise-grade security. ## Comparison Table | Feature | TestingBot | Browserbase | Steel.dev | Browserless | Lightpanda | | --- | --- | --- | --- | --- | --- | | Real Browsers & Devices | ✅ Yes | ❌ No | ❌ No | ❌ No | ❌ No | | Headless Browsers | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | | Global Testing Locations | ✅ Yes | ✅ Yes | ❌ No | ❌ No | ❌ No | | Parallel Sessions | ✅ Unlimited | ❌ Limited | ✅ Yes | ❌ Limited | ❌ Limited | | Security & Compliance | ✅ Enterprise-Grade | ❌ Basic | ❌ Basic | ❌ Basic | ❌ Basic | ## AI Integrations TestingBot is capable of connecting remote browsers to various AI frameworks and libraries, including the following popular options: - [AgentKit](https://testingbot.com/support/ai/integrations/agentkit) - Create AI agents that can run browser automation tasks using TestingBot's remote browser grid and AgentKit. - [Browser-Use](https://testingbot.com/support/ai/integrations/browser-use) - A Python library for browser automation using AI agents. - [Braintrust](https://testingbot.com/support/ai/integrations/braintrust) - Create tools to leverage AI for testing and automation. - [Stagehand](https://testingbot.com/support/ai/integrations/stagehand) - An AI Browser Automation Framework that allows you to create and manage browser automation tasks using AI. - [Magnitude](https://testingbot.com/support/ai/integrations/magnitude) - An AI framework using Vision to perform automated browser automation with LLMs. - [Notte](https://testingbot.com/support/ai/integrations/notte) - A reliable and fast AI browser agent. ## Scraping data with real browsers Looking to obtain data from the world wide web with real browsers? You can connect your [Puppeteer](https://testingbot.com/support/web-automate/puppeteer) and [Playwright](https://testingbot.com/support/web-automate/playwright) scripts to the large TestingBot browser grid, or use functions to perform tasks without writing any code: - [Take screenshots](https://testingbot.com/support/functions/screenshot) - [Generate PDFs](https://testingbot.com/support/functions/pdf) - [Scrape Webpages](https://testingbot.com/support/functions/scrape) - [Serverless Functions](https://testingbot.com/support/functions/serverless) ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/ai/integrations/agentkit # Using AgentKit with TestingBot Create AI agents that can run browser automation tasks using TestingBot's remote browser grid and AgentKit (by inngest). ## Dependencies To install all dependencies, please follow these steps: [npm](https://testingbot.com#) [pnpm](https://testingbot.com#) [yarn](https://testingbot.com#) ```bash npm install @inngest/agent-kit playwright-core ``` ```bash pnpm add @inngest/agent-kit playwright-core ``` ```bash yarn add @inngest/agent-kit playwright-core ``` ## Create a TestingBot service In the example below, we'll show an example tool that will use Algolia to search Hacker News (HN) posts. The TestingBot remote browser grid will be used to perform the actions and return the results. The example below uses the `chromium.connect` method to connect to a remote browser in the TestingBot cloud. There are a lot of [Playwright Options](https://testingbot.com/support/web-automate/playwright/options) you can use to customize the browser session. Make sure to replace `key` and `secret` with your TestingBot API key and secret. ```javascript import "dotenv/config"; import { anthropic, createAgent, createNetwork, createTool, } from "@inngest/agent-kit"; import { createServer } from "@inngest/agent-kit/server"; import { z } from "zod"; import { chromium } from "playwright-core"; const testingBotKey = process.env.TESTINGBOT_KEY; const testingBotSecret = process.env.TESTINGBOT_SECRET; const searchHN = createTool({ name: "search_hn", description: "Search HN posts and comments", parameters: z.object({ query: z.string().describe("The search query for HN"), }), handler: async ({ query }, { step }) => { return await step?.run("search-on-hn", async () => { const browser = await chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?key=${testingBotKey}&secret=${testingBotSecret}&browserName=chrome&browserVersion=latest`, }); try { const page = await browser.newPage(); // Construct the search URL const searchUrl = `https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=${query}&sort=byPopularity&type=story`; await page.goto(searchUrl); // Wait for results to load await page.waitForSelector(".SearchResults", { timeout: 10000 }); // Extract search results const results = await page.evaluate(() => { const posts = document.querySelectorAll(".SearchResults .Story"); return Array.from(posts).map((post) => ({ title: post.querySelector(".Story_title span")?.textContent?.trim(), })); }); return results.slice(0, 5); // Return top 5 results } finally { await browser.close(); } }); }, }); // Create the search agent const searchAgent = createAgent({ name: "hn_searcher", description: "An agent that searches HN for relevant information", system: "You are a helpful assistant that searches HN through algolia for relevant information.", tools: [searchHN], }); // Create the network const hnSearchNetwork = createNetwork({ name: "hn-search-network", description: "A network that searches HN using TestingBot", agents: [searchAgent], maxIter: 2, defaultModel: anthropic({ model: "claude-sonnet-4-5", defaultParameters: { max_tokens: 4096, }, }), }); // Create and start the server const server = createServer({ networks: [hnSearchNetwork], }); server.listen(3010, () => console.log("HN Search Agent server is running on port 3010") ); ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/ai/integrations/braintrust # Using BrainTrust with TestingBot ## Braintrust Organization To get started, make sure you create [a new Braintrust organization](https://www.braintrust.dev/docs/guides/projects). Next, install the necessary dependencies: ## Install dependencies To use Braintrust with TestingBot, you need to install the following dependencies: ```bash npm install zod braintrust playwright-core ``` ## Variables To use TestingBot with Braintrust, you need to set the following environment variables: `TESTINGBOT_KEY` and `TESTINGBOT_SECRET`. You will also need to set up the AI API Key that you are using. Braintrust supports various AI providers, such as OpenAI, Anthropic and others. ## Tools Braintrust allows you to create [tools](https://www.braintrust.dev/docs/guides/functions/tools) that can be used by agents. In this example, we will create a tool that takes a URL parameter and will use Readability to convert the HTML. ```javascript import * as braintrust from "braintrust"; import { z } from "zod"; import { chromium } from "playwright-core"; const testingBotKey = process.env.TESTINGBOT_KEY; const testingBotSecret = process.env.TESTINGBOT_SECRET; async function loadPage({ url }: { url: string }) { const browser = await chromium.connectOverCDP( `wss://cloud.testingbot.com/?key=${testingBotKey}&secret=${testingBotSecret}&browserName=chrome&browserVersion=latest` ); const defaultContext = browser.contexts()[0]; const page = await defaultContext.newPage(); await page.goto(url); const readable = await page.evaluate(async () => { const readability = await import('https://cdn.skypack.dev/@mozilla/readability'); return new readability.Readability(document).parse(); }); const text = `${readable?.title ?? ''}\n${readable?.textContent ?? ''}`; await browser.close(); return { page: text }; } const project = braintrust.projects.create({ name: "TestingBot API Tool" }); project.tools.create({ handler: loadPage, parameters: z.object({ url: z.string(), }), returns: z.object({ page: z.string(), }), name: "Load page", slug: "load-page", description: "Load a page from the internet", ifExists: "replace", }); ``` ## Deploy tool You can now deploy the tool to Braintrust. You can do this by running the following command: ```bash npx braintrust push testingbot.ts ``` Once the tool is deployed, you will see the tool listed in the Library's Tools section. Now you can specify a URL and use the "Load page" tool to use the TestingBot tool. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/ai/integrations/browser-use # Browser Use Cloud API Browser-Use is a Python library that allows you to create browser automation scripts using AI. You can use multiple models such as GPT-4o, DeepSeek, Gemini-2.0, Claude and more. Local models such as Qwen 2.5 are also supported. The library essentially uses a Playwright browser instance to execute tasks on a (remote) browser. You can perform tasks such as: - Navigate to Google and search for TestingBot - Go to Reddit and open an article - Open a new tab and go to CNN By using the TestingBot remote browser grid, you can run multiple AI agents in parallel without worrying about setting up and managing the browsers yourself. Additionally, TestingBot records each session with a video, logs and generated metadata are saved as well. You can run browsers from [various geographical locations](https://testingbot.com/support/web-automate/playwright/options#geo) and use different browser versions, on multiple operating systems. To install browser use, please follow these steps: - Install the Browser-Use library: `pip install browser-use` - Install the Playwright library: `pip install playwright` ## Connecting to a remote browser Please see the example below on how to connect your Browser-Use agent to a remote browser in the TestingBot cloud. The `wss` URL points to the TestingBot cloud, where you can specify the browser and platform you want to use. You can also specify the name of the session. Make sure to replace `key` and `secret` with your TestingBot API key and secret. ```python import json import httpx import os import asyncio import logging from browser_use.browser import BrowserSession from browser_use.agent.service import Agent from browser_use.llm import ChatOpenAI capabilities = { "tb:options": { "name": "Browser Use Example", "key": os.environ["TESTINGBOT_KEY"], "secret": os.environ["TESTINGBOT_SECRET"], }, "browserName": "chrome", "browserVersion": "latest", "platform": "WIN10" } async def create_testingbot_session(): try: async with httpx.AsyncClient(timeout=600.0) as client: response = await client.post( "https://cloud.testingbot.com/session", json={ "capabilities": capabilities }, ) response.raise_for_status() session_data = response.json() logging.info( "TestingBot session created successfully: %s", json.dumps(session_data, indent=2) ) return session_data except Exception as e: logging.error("Failed to create TestingBot session: %s", e) async def main(): session_data = await create_testingbot_session() cdp_url = session_data["cdp_url"] browser_session = BrowserSession(cdp_url=cdp_url) agent = Agent( task="Go to the TestingBot homepage and click the pricing menu item", llm=ChatOpenAI(model="gpt-4o"), browser_session=browser_session, ) try: await agent.run() finally: await browser_session.close() if __name__ == " __main__": asyncio.run(main()) ``` When you run this example, the Browser-Use agent will connect to a remote browser in the TestingBot cloud, and execute the task specified in the `task` variable. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/ai/integrations/stagehand # TestingBot & Stagehand Stagehand is an AI Browser Automation Framework that allows you to create and manage browser automation tasks using AI. It is available in both NodeJS and Python. By connecting Stagehand to TestingBot's remote browser grid, you can run multiple AI-driven browser automation tasks in parallel, without the need to manage the browsers yourself. Each session is video recorded and available for replay in the TestingBot dashboard. You can run browsers from [various geographical locations](https://testingbot.com/support/web-automate/playwright/options#geo) and use different browser versions, on multiple operating systems. ## Connecting to a remote browser To get started, please install these packages: - Install the Stagehand library: `@browserbasehq/stagehand` - Install the TestingBot library: `npm install testingbot-api` We can now run a simple example. This will connect Stagehand with a remote TestingBot browser session. Use the AI prompt to test a specific scenario, by describing what you want to test in your own natural language. ```javascript const { Stagehand } = require("@browserbasehq/stagehand"); const TestingBot = require("testingbot-api"); (async () => { const tb = new TestingBot({ api_key: process.env.TESTINGBOT_KEY, api_secret: process.env.TESTINGBOT_SECRET }); const options = { capabilities: { browserName: 'chrome', browserVersion: 'latest', platform: 'WIN11' } }; const session = await tb.createSession(options); const stagehand = new Stagehand({ env: "LOCAL", localBrowserLaunchOptions: { cdpUrl: session['cdp_url'] } }); await stagehand.init(); const page = stagehand.page; await page.goto("https://testingbot.com/pricing"); const extractResult = await page.extract("Extract the pricing plans from this page"); console.log(`Extract result:\n`, extractResult); })(); ``` Make sure to replace `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` with your actual TestingBot API key and secret. In the example above, we create a new TestingBot session with the desired capabilities. We then pass the `cdpUrl` to Stagehand, which allows it to connect to the remote browser session on TestingBot. You can now use Stagehand's AI capabilities to interact with the page, extract information, or perform other actions as needed. For more information on using Stagehand, please refer to the [Stagehand documentation](https://docs.stagehand.dev/first-steps/quickstart). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/ai/integrations/magnitude # Magnitude AI [Magnitude](https://github.com/magnitudedev/magnitude) is an AI browser automation framework that allows you to run browser automation scripts using AI models. It uses Vision AI to understand the webpage and interact with it, making it possible to automate tasks that require visual understanding. You can use the framework to automate tasks on webpages, test functionality, extract data and verify any specific behavior on the webpage, just like a normal user would do. ## Running Magnitude AI tests on TestingBot To run Magnitude AI tests on TestingBot, you can use the following steps and example code to get started: - `npx create-magnitude-app` - Next, modify the `src/index.tsx` file that was generated, add a `browser` option to the `startBrowserAgent` function. ```javascript import { startBrowserAgent } from "magnitude-core"; import z from 'zod'; import dotenv from 'dotenv'; dotenv.config(); const capabilities = { 'tb:options': { key: process.env.TESTINGBOT_KEY, secret: process.env.TESTINGBOT_SECRET, }, browserName: 'chrome', browserVersion: 'latest' } async function main() { const agent = await startBrowserAgent({ // Starting URL for agent url: 'https://docs.magnitude.run/getting-started/quickstart', // Show thoughts and actions narrate: true, browser: { cdp: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` } }); // Intelligently extract data based on the DOM content matching a provided zod schema const gettingStarted = await agent.extract('Extract how to get started with Magnitude', z.object({ // Agent can extract existing data or new insights difficulty: z.enum(['easy', 'medium', 'hard']), steps: z.array(z.string()), })); // Navigate to a new URL await agent.nav('https://magnitasks.com'); // Magnitude can handle high-level tasks await agent.act('Create a task', { // Optionally pass data that the agent will use where appropriate data: { title: 'Get started with Magnitude', description: gettingStarted.steps.map(step => `• ${step}`).join('\n') } }); // It can also handle low-level actions await agent.act('Drag "Get started with Magnitude" to the top of the in progress column'); // Stop agent and browser await agent.stop(); } main(); ``` When you run this example, the Magnitude AI agent will use a remote TestingBot browser to interact with. Results will appear on your machine. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/ai/integrations/notte # Notte Browser Agent [Notte](https://github.com/nottelabs/notte) is Browser AI agent that allows you to run browser automation scripts using AI models. It is a web agent framework built for speed, cost-efficiency, scale and reliability. You can use the framework to automate tasks on webpages, test functionality, extract data and verify any specific behavior on the webpage, just like a normal user would do. ## Running Notte Browser Agent tasks with TestingBot Notte allows you to specify a `CDP` endpoint, enabling it to connect to remote browser instances. To run Notte Browser Agent tasks on TestingBot, you can use the following steps and example code to get started: - `pip install notte 'litellm[proxy]' python-dotenv` - Add your Gemini API credentials and TestingBot credentials to the `.env` file: ```bash GEMINI_API_KEY="..." TESTINGBOT_KEY="api_key" TESTINGBOT_SECRET="api_secret" ``` ```python import os import notte from dotenv import load_dotenv load_dotenv() key = os.environ["TESTINGBOT_KEY"] secret = os.environ["TESTINGBOT_SECRET"] cdp_url = f"wss://cloud.testingbot.com/?key={key}&secret={secret}&browserName=chrome&browserVersion=latest" with notte.Session(cdp_url=cdp_url) as session: agent = notte.Agent(session=session, reasoning_model='gemini/gemini-2.5-flash', max_steps=30) response = agent.run(task="extract pricing plans from https://testingbot.com") ``` You can find more CDP options on [this page](https://testingbot.com/support/web-automate/puppeteer/options). When you run this example, a remote browser session will be started on TestingBot. Notte will use the Gemini LLM to interact with the remote browser and run the task you specified. By using TestingBot as the CDP endpoint, you can run Notte Browser Agent tasks on any of the available browsers in the TestingBot cloud, with speed, scale and reliability. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/ai/chat # What is TestingBot AI Chat? TestingBot AI Chat is an innovative, codeless test automation tool that harnesses Generative AI and Large Language Models (LLMs) to let users interact with an AI agent in a chat-based interface. Instead of writing complex scripts or manually locating elements, simply type a prompt and the AI agent will operate a remote browser in real time. Your prompts can be transformed into automated test cases, allowing you to reuse them for continuous testing of your website or web application. Additionally, the AI agent can generate Selenium WebDriver, Playwright or Puppeteer test scripts based on its recorded interactions with the remote browser. To get started, upgrade your account and go to the [TestingBot member dashboard](https://testingbot.com/members). Click **AI Chat** in the left-hand menu to launch the AI Chat interface. https://www.youtube.com/embed/dEu7sKL0pMo?si=vaAP1L51M37z244y&rel=0 ## Example Prompts Below are some example prompts that you can use with TestingBot AI Chat: - ``` Navigate to https://cnn.com and click on the first article. Verify that it loads correctly. Generate a Playwright test for this. ``` - ``` Go to https://testingbot.com/pricing and click on the Enterprise plan. Verify that the pricing details are correct. Generate a Puppeteer test for this. ``` - ``` Open a new tab and go to https://github.com. Search for "TestingBot" and click on the first result. Verify that the page loads correctly. Generate a Selenium WebDriver test for this. ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/ai/codeless # AI Codeless Web Testing TestingBot provides AI solutions for QA and developers to automate their tests and other tasks. Simply instruct our AI agent what to test and the AI agent will create and run tests against your website on regular intervals, alerting you if a test fails. Enter a URL and a prompt to instruct the AI agent what to test. The AI agent will run the test on a real browser in the TestingBot cloud. When an error is detected, the AI agent will alert you. Instead of generating test code from ChatGPT or Claude, you can instruct the AI agent to run tests based on natural language prompts. Unlike traditional test scripts, which can break as soon as the website's UI changes, AI-driven tests dynamically interpret your instructions each time they run. This ensures your tests remain adaptable and up-to-date, even as the website evolves. ## Getting Started To get started, please go to the [Codeless Testing](https://testingbot.com/members/lab) section and click the **Create a new AI test** button. [Create a new AI test](https://testingbot.com/members/lab) ![Add AI test](https://testingbot.com/assets/support/ai/codeless/create_test-d1486a45a5a31b82ae154681a1591bdf167500f8def01722e2d61abb6858559e.png) ### Create a new test You will see the form below, where you can add a name for the test and the webpage you want to start the test with. In the textarea you can write the test prompt in your own language. You can find example prompts in the next [prompts section](https://testingbot.com#prompts). ## Prompts The AI agent will run tests based on the prompts you provide. A prompt is a set of instructions for the AI agent to follow. We recommend keeping test prompts as concise as possible. For example, you can instruct the AI agent to: - Click on a button - Fill in a form - Test a checkout flow - Verify that an element is visible or not visible - Verify that an element contains specific text - Verify that an element does not contain specific text - Verify that an element has or does not have a specific attribute ### Example Prompts Below are some example prompts you could use to test a website: #### User Authentication & Account Management - Click on the "Sign In" button and verify that the login form appears. - Try logging in with an incorrect password and confirm that an error message appears. #### Forms & Input Validation - Fill in the registration form with valid data and submit it. - Try submitting the registration form without entering required fields and verify that validation errors appear. - Enter an invalid email format and check that an error message is displayed. #### Navigation & UI Elements - Click on the "Home" link in the navigation menu and ensure it redirects correctly. - Open the mobile menu and confirm that all expected links are visible. - Scroll down the homepage and check that the footer loads correctly. - Hover over the dropdown menu and verify that all submenu items appear. #### E-Commerce & Payment Testing - Add a product to the cart and verify that the cart updates. - Enter invalid credit card details and ensure an error message appears. #### Content & Accessibility - Check that all images on the homepage load properly. - Verify that all links on the website are clickable and lead to the correct pages. - Use keyboard navigation to tab through elements and ensure accessibility compliance. - Enable dark mode (if available) and confirm that the UI adjusts correctly. - Resize the browser window and confirm that the website is responsive. #### Performance & Security - Try opening the website in an incognito window and verify that session-based content does not persist. - Attempt to submit a form with script injection and confirm that it is sanitized. - Check if the website forces HTTPS and does not allow unsecured connections. #### Multilingual & Localization Testing - Switch the website language to French and confirm that all text updates correctly. - Change the currency on the pricing page and verify that the prices are updated. ## Browsers You can select one or more browsers to run your tests on. The AI agent will run the tests on the selected browsers. Run your tests on the latest versions of Chrome and Edge, on Windows and Linux. ![Test on Browsers with AI](https://testingbot.com/assets/support/ai/browsers-0755c758094f63992df0c601d5083f6d1990c3dc56af52baf404da9e4a06b8d3.png) ## Schedule Indicate when and at which interval, you would like to run the test. You can choose to run the test once, on a daily basis, weekly, or repeat every xx minutes/hours. ![Schedule AI browser tests](https://testingbot.com/assets/support/ai/schedule-6ba4a52ae1422505b0f905a58d5aa486c51948981cb6de822d9a4f6ca0238d89.png) ## Alerting When a test fails, you can choose to be alerted via e-mail, API call (webhook) or SMS text message. ![Schedule AI browser alerts](https://testingbot.com/assets/support/ai/alerting-5f059647c254b9279372ad0b1e23f3319721d940595838cb56ec0a0c1834bd1a.png) ## API You can use the TestingBot REST-API to [create AI tests from prompts](https://testingbot.com/support/api/codeless#createtest). Create one or more tests with your prompt, assign browsers and set a schedule. Use [webhooks](https://testingbot.com/support/integrations/webhooks) to get notified of the results. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/ai/mcp # What is TestingBot MCP? The TestingBot MCP server (Model Context Protocol) is a small program that runs locally on your machine and connects your AI agent or Large Language Model to TestingBot's testing infrastructure. Your AI client sends natural-language requests, and the server turns them into actions: it calls the TestingBot API to manage your account and tests, and it drives real desktop browsers, mobile browsers and native apps in the cloud. It works with any MCP-compatible client (Claude, Cursor, VS Code, Windsurf and more) and is model-agnostic. You do not write any test code or call any API directly. You describe what you want, and the agent picks the right tools. Looking to test the WebMCP tools your own website exposes to in-browser AI agents? That is a different feature. See [Testing WebMCP Tools](https://testingbot.com/support/ai/webmcp). ## What you can do The TestingBot MCP server ships as two bundled packages that together expose a broad set of tools. They fall into three broad areas. ### Account & test management Query and manage everything in your TestingBot account through conversation, without opening the dashboard: - **Tests** : list recent tests, read full session details, update metadata, mark passed/failed, stop or delete tests. - **Builds** : list builds, see the tests inside a build, and delete builds. - **Browsers & devices** : list available desktop browsers, platforms and mobile devices (real and emulators/simulators). - **Screenshots** : capture a URL across multiple browsers and retrieve the resulting images. - **Storage** : upload native mobile apps (`.apk`/`.ipa`/`.zip`) for testing. - **Team & user** : view account info, team settings and usage, and update your profile. - **Tunnels & logs** : list/stop tunnels and fetch session failure logs for debugging. - **Project scaffolding** : scan a project for test files and generate a ready-to-paste TestingBot config for your framework. ### App Automate: Maestro, Espresso and XCUITest The agent can run a full native-app suite end to end, without you touching a CI config. It uploads the build, uploads (or writes) the tests, picks a device, starts the run and reports back: - **[Maestro](https://testingbot.com/support/app-automate/maestro)**: upload an app and its flow YAML (from a directory, a zip, or written inline by the agent), run it on real devices, emulators or simulators, filter by tag, shard across devices, then pull per-flow results, video, screenshots and device logs. Failed flows can be fixed and retried individually. - **[Espresso](https://testingbot.com/support/app-automate/espresso)**: run a pre-built instrumented Android suite, optionally narrowed to specific test classes, packages or annotations. - **[XCUITest](https://testingbot.com/support/app-automate/xcuitest)**: run a zipped XCUITest runner bundle against your `.ipa` on iOS devices. All three are asynchronous: the run starts, the agent polls it, and fetches the JUnit report and failure detail when it finishes. Nothing blocks while devices execute. ### Live browser & mobile automation Beyond account management, the bundled automation package lets the agent drive real sessions step by step: - **Desktop browsers** : open real Chrome, Firefox, Edge, Safari or IE on Windows, macOS and Linux, then navigate, click, type, read text, run JavaScript and take screenshots. - **Mobile browsers** : drive Chrome/Safari on Android and iOS, on emulators/simulators (the cheaper default) or on physical real devices. - **Native mobile apps** : launch and control uploaded iOS/Android apps with Appium. - **Interactive live sessions** : start a manual live testing session, or watch and take over any automated session via its live-view URL. - **CDP sessions** : get a remote browser with a Chrome DevTools Protocol endpoint to drive with Playwright or Puppeteer. For the complete list of tools and their parameters, see the [Tool reference](https://testingbot.com/support/ai/mcp/tools). For end-to-end examples, see [Use cases & workflows](https://testingbot.com/support/ai/mcp/use-cases). ## How it works The MCP server runs as a local process, started by your AI client over standard input/output. When you type a request, your AI client decides which tools to call and sends those calls to the server. The server then does one of three things: 1. For account and test operations, it makes authenticated calls to the TestingBot API and returns the results to your client in a readable format. 2. For automation, it spins up a real cloud session (desktop or mobile) and issues WebDriver/Appium commands against it, returning a `sessionId` and a live-view URL so you can follow along. 3. For App Automate runs (Maestro, Espresso, XCUITest), it uploads your app and tests, queues the run on TestingBot's device cloud and hands back a run id. The run executes on our infrastructure while your agent polls it, so a long suite never ties up the conversation. The MCP server is distributed as `@testingbot/mcp-server`, which handles account and orchestration tools and bundles `@testingbot/automation-mcp` for live browser and mobile automation. You install just the one package, both sets of tools become available automatically. Every automation session returns a live-view URL in the form `https://testingbot.com/tests//live?auth=` so you can watch the session run, or step in manually. Your TestingBot credentials stay on your local machine. The MCP server talks directly to the TestingBot API, your keys are never sent to the LLM provider. Read more on the [Authentication](https://testingbot.com/support/ai/mcp/authentication) page. ## Supported clients The TestingBot MCP server works with any MCP-compatible client through a standard stdio configuration. Tested clients include: - Claude Desktop and Claude Code - Cursor - VS Code (Copilot / MCP) - Windsurf - Any other MCP-compatible client (generic stdio config) 1-click installers are available for VS Code and Cursor, and Claude can install the packaged `.mcpb` bundle. See the cards in the [Quick start](https://testingbot.com#quick-start) below, or the full [Setup & installation](https://testingbot.com/support/ai/mcp/setup) guide for per-client configuration. ## Quick start Getting up and running takes three steps. ### 1. Install Use one of the 1-click installers below, or add the server to your client's MCP configuration manually. The server runs through `npx`, so there is nothing to install globally: ```bash npx -y @testingbot/mcp-server@latest ``` In your MCP client, that maps to the standard configuration block: ```json { "mcpServers": { "testingbot": { "command": "npx", "args": ["-y", "@testingbot/mcp-server@latest"], "env": { "TESTINGBOT_KEY": "", "TESTINGBOT_SECRET": "" } } } } ``` For 1-click setup, pick your editor below: #### Visual Studio Code Install the TestingBot MCP server for Visual Studio Code with a single click. This will automatically configure the MCP settings in your VS Code environment. [Install for VS Code](https://testingbot.com/mcp/install?client=vscode) #### Cursor Install the TestingBot MCP server for Cursor with a single click. This will automatically configure the MCP settings in your Cursor editor. [Install for Cursor](https://testingbot.com/mcp/install?client=cursor) #### Claude Install the `mcp-server.mcpb` file from the [MCP releases page](https://github.com/testingbot/mcp-server/releases). Claude will automatically set up the TestingBot MCP server. For per-client configuration file locations and step-by-step instructions for every supported client, see the full [Setup & installation](https://testingbot.com/support/ai/mcp/setup) guide. ### 2. Authenticate Once the server is configured, connect it to your TestingBot account. The recommended way is the `tb_login` tool, just ask your agent to log in, and a browser window opens for you to authorize. Nothing is pasted, and your credentials are written to `~/.testingbot/credentials` on your machine and used automatically afterwards. Alternatively, set `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` in the client's `env` block (shown in the config above), useful for CI and shared configs. Get your keys from your [account settings](https://testingbot.com/members/user/edit). See [Authentication](https://testingbot.com/support/ai/mcp/authentication) for browser login, device-code login for remote/headless machines, and credential storage details. ### 3. Ask your agent That is it. Describe what you want in plain language and the agent will pick the right tools. For example: > "Open Chrome on Windows 11, go to https://example.com, and tell me the page title." > "List my last 10 TestingBot tests and show me the details of any that failed." > "Run the Maestro flows in ./flows against ./app.apk on a real Pixel 8, then tell me which ones failed and why." Browse many more end-to-end examples in [Use cases & workflows](https://testingbot.com/support/ai/mcp/use-cases). ## Documentation index Dive deeper into any part of the TestingBot MCP server: - [**Setup & installation**](https://testingbot.com/support/ai/mcp/setup): prerequisites, install methods and per-client configuration for Claude, Cursor, VS Code, Windsurf and more. - [**Authentication**](https://testingbot.com/support/ai/mcp/authentication): browser login, device-code flow, API keys, credential storage and security. - [**Tool reference**](https://testingbot.com/support/ai/mcp/tools): every tool, grouped by category, with full parameter tables. - [**Use cases & workflows**](https://testingbot.com/support/ai/mcp/use-cases): real scenarios with example prompts and the tool sequences behind them. - [**Troubleshooting**](https://testingbot.com/support/ai/mcp/troubleshooting): fixes for common issues with login, devices, screenshots and sessions. - [**FAQ**](https://testingbot.com/support/ai/mcp/faq): answers to the most common questions about the MCP server. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/ai/mcp/setup # Setup & installation This guide walks you through installing the TestingBot MCP server and connecting it to your AI client. The server runs locally and is started on demand by your client through `npx`, so there is nothing to install globally and nothing to keep running yourself. New to TestingBot MCP? Start with the [overview](https://testingbot.com/support/ai/mcp) to understand what the server can do. For everything about logging in, see the dedicated [Authentication](https://testingbot.com/support/ai/mcp/authentication) page. ## Prerequisites Before you begin, make sure you have the following: - **Node.js 18 or higher** (20+ recommended). The server is launched with `npx`, which ships with Node.js. Check your version with `node -v`. - **An MCP-compatible client** such as Claude Desktop, Claude Code, Cursor, VS Code (Copilot/MCP), Windsurf, or any other client that supports the Model Context Protocol over stdio. - **A TestingBot account**. You can [sign up for free](https://testingbot.com/users/sign_up) if you do not have one yet. You do **not** need to paste an API key to get started. The recommended way to sign in is the `tb_login` tool, which opens a browser and captures your credentials without copy/paste. Authentication is covered in full on the [Authentication](https://testingbot.com/support/ai/mcp/authentication) page. ## Install The TestingBot MCP server is distributed as the `@testingbot/mcp-server` NPM package. It bundles `@testingbot/automation-mcp`, which adds live browser and mobile automation tools, so a single install gives you both account management and automation capabilities. You do not install the package by hand. Instead you add a small configuration entry to your MCP client that tells it to launch the server with `npx`. The command your client will run is: ```bash npx -y @testingbot/mcp-server@latest ``` The `-y` flag lets `npx` download the package the first time without prompting, and `@latest` ensures you always start the most recent version. Keep `@latest` rather than pinning a version: newer releases add tools, and a pinned older version silently lacks them. App Automate (Maestro, Espresso and XCUITest) needs 1.1.0 or later, and a `.mcpb` bundle installed in Claude Desktop has to be re-downloaded to update. ### 1-click installers For VS Code and Cursor you can skip the manual configuration with a 1-click installer, which writes the MCP config for you: - [Install for VS Code](https://testingbot.com/mcp/install?client=vscode) - [Install for Cursor](https://testingbot.com/mcp/install?client=cursor) For Claude Desktop you can install the `.mcpb` bundle from the [MCP server releases page](https://github.com/testingbot/mcp-server/releases), Claude will configure the server automatically. The standard configuration below is identical for every client, only the location of the config file changes. Pick your client in the tabs in the next section. ## Per-client configuration Select your client below. Each tab shows where the configuration file lives and the JSON to add. Two variants are shown for each client: - **Browser login (recommended)**: no key in your config. After the server starts, ask your agent to run `tb_login` and authorize in the browser. - **API key** : set `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` in the `env` block. Best for CI and shared configurations. Get your keys at [your account settings](https://testingbot.com/members/user/edit). [Claude Desktop](https://testingbot.com#) [Claude Code](https://testingbot.com#) [Cursor](https://testingbot.com#) [VS Code](https://testingbot.com#) [Windsurf](https://testingbot.com#) [Other](https://testingbot.com#) Edit Claude Desktop's `claude_desktop_config.json`. You can open it from **Settings → Developer → Edit Config** , or find it at: - macOS: `~/Library/Application Support/Claude/claude_desktop_config.json` - Windows: `%APPDATA%\Claude\claude_desktop_config.json` **Browser login (recommended)**, no key, run `tb_login` afterwards: ```json { "mcpServers": { "testingbot": { "command": "npx", "args": ["-y", "@testingbot/mcp-server@latest"] } } } ``` **API key** variant: ```json { "mcpServers": { "testingbot": { "command": "npx", "args": ["-y", "@testingbot/mcp-server@latest"], "env": { "TESTINGBOT_KEY": "", "TESTINGBOT_SECRET": "" } } } } ``` Save the file and restart Claude Desktop. The TestingBot tools will appear in the tools menu. Claude Code reads MCP servers from a `.mcp.json` file in your project root (or your user config). The quickest way is the CLI: ```bash claude mcp add testingbot -- npx -y @testingbot/mcp-server@latest ``` To configure it by hand instead, create or edit `.mcp.json` in your project root. **Browser login (recommended)**, no key, run `tb_login` afterwards: ```json { "mcpServers": { "testingbot": { "command": "npx", "args": ["-y", "@testingbot/mcp-server@latest"] } } } ``` **API key** variant: ```json { "mcpServers": { "testingbot": { "command": "npx", "args": ["-y", "@testingbot/mcp-server@latest"], "env": { "TESTINGBOT_KEY": "", "TESTINGBOT_SECRET": "" } } } } ``` Run `/mcp` inside Claude Code to confirm the `testingbot` server is connected. Use the [1-click installer for Cursor](https://testingbot.com/mcp/install?client=cursor), or configure it manually by creating or opening `.cursor/mcp.json` in your project (or `~/.cursor/mcp.json` for a global config). **Browser login (recommended)**, no key, run `tb_login` afterwards: ```json { "mcpServers": { "testingbot": { "command": "npx", "args": ["-y", "@testingbot/mcp-server@latest"] } } } ``` **API key** variant: ```json { "mcpServers": { "testingbot": { "command": "npx", "args": ["-y", "@testingbot/mcp-server@latest"], "env": { "TESTINGBOT_KEY": "", "TESTINGBOT_SECRET": "" } } } } ``` Restart Cursor. The TestingBot server will appear under **Settings → MCP**. Use the [1-click installer for VS Code](https://testingbot.com/mcp/install?client=vscode), or configure it manually. Open the command palette (`Ctrl+Shift+P` / `Cmd+Shift+P`), run **MCP: Add Server** , and choose the NPM package method with `@testingbot/mcp-server`. You can also edit the workspace file `.vscode/mcp.json` directly. Note that VS Code nests servers under a `servers` key: **Browser login (recommended)**, no key, run `tb_login` afterwards: ```json { "servers": { "testingbot": { "command": "npx", "args": ["-y", "@testingbot/mcp-server@latest"] } } } ``` **API key** variant: ```json { "servers": { "testingbot": { "command": "npx", "args": ["-y", "@testingbot/mcp-server@latest"], "env": { "TESTINGBOT_KEY": "", "TESTINGBOT_SECRET": "" } } } } ``` Reload the window. The TestingBot tools become available to Copilot Chat in agent mode. Windsurf reads MCP servers from `~/.codeium/windsurf/mcp_config.json`. You can also open it from **Settings → Cascade → MCP Servers → Manage MCPs**. **Browser login (recommended)**, no key, run `tb_login` afterwards: ```json { "mcpServers": { "testingbot": { "command": "npx", "args": ["-y", "@testingbot/mcp-server@latest"] } } } ``` **API key** variant: ```json { "mcpServers": { "testingbot": { "command": "npx", "args": ["-y", "@testingbot/mcp-server@latest"], "env": { "TESTINGBOT_KEY": "", "TESTINGBOT_SECRET": "" } } } } ``` Refresh the MCP servers list in Windsurf to connect. Any MCP-compatible client can launch the server over stdio. Add an entry that runs the standard command. Most clients use the `mcpServers` shape below: **Browser login (recommended)**, no key, run `tb_login` afterwards: ```json { "mcpServers": { "testingbot": { "command": "npx", "args": ["-y", "@testingbot/mcp-server@latest"] } } } ``` **API key** variant: ```json { "mcpServers": { "testingbot": { "command": "npx", "args": ["-y", "@testingbot/mcp-server@latest"], "env": { "TESTINGBOT_KEY": "", "TESTINGBOT_SECRET": "" } } } } ``` If your client uses a different key (for example `servers` instead of `mcpServers`), keep the `command`, `args` and optional `env` values exactly as shown and adapt the wrapper to your client's format. We recommend the browser-login variant: it keeps long-lived API secrets out of config files. After the server starts, ask your agent to run `tb_login`. See [Authentication](https://testingbot.com/support/ai/mcp/authentication) for the full flow, including the device-code option for SSH and headless environments. If you use the API-key variant, treat your config file like a secret. Avoid committing files that contain `TESTINGBOT_SECRET` to source control. ## Verify your setup Once your client has loaded the server, confirm everything works by asking your AI agent to perform a simple read-only action: > "List my recent TestingBot tests" > "Show my TestingBot account info" Behind the scenes these run the `getTests` and `getUserInfo` tools. A successful response lists your tests, or returns account details such as your name, plan, company, country, available seconds and concurrency. ### What success looks like - The TestingBot tools appear in your client's tool list (often shown as a tools or MCP icon). - The agent returns real account data rather than an error. - No restart is needed after authenticating, credentials are picked up automatically. ### If you see “Run tb\_login” If a tool replies with a message like _“No TestingBot credentials configured. Run the tb\_login tool…”_, the server started successfully but is not yet authenticated (this is the expected state for the browser-login variant). Ask your agent to run `tb_login` and complete the sign-in: > "Run tb\_login to authenticate with TestingBot" On a desktop this opens your browser so you can click **Authorize**. On a remote or headless host, it prints a short code, open [https://testingbot.com/device](https://testingbot.com/device), enter the code, and authorize. Full details are on the [Authentication](https://testingbot.com/support/ai/mcp/authentication) page. Run into a problem? The [Troubleshooting](https://testingbot.com/support/ai/mcp/troubleshooting) page covers common issues such as Node version errors, the server not appearing, and credential prompts. ## Next steps Now that the server is installed and verified, continue with: - [Authentication](https://testingbot.com/support/ai/mcp/authentication), browser login, device-code flow, API keys and where credentials are stored. - [Tool reference](https://testingbot.com/support/ai/mcp/tools), every tool the server exposes, grouped by category. - [Use cases & workflows](https://testingbot.com/support/ai/mcp/use-cases), end-to-end examples and the prompts that drive them. - [FAQ](https://testingbot.com/support/ai/mcp/faq), quick answers to common questions. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/ai/mcp/authentication # Authentication Before the TestingBot MCP server can manage tests, drive real browsers or query your account, it needs your TestingBot credentials. There are three ways to authenticate, and the right one depends on where you run your MCP client, a desktop machine, an SSH session, a CI pipeline or a hosted web client. This page covers all three methods, where credentials are stored, the full list of environment variables you can set, and how the server behaves when no credentials are present. If you have not installed the server yet, start with [Setup & installation](https://testingbot.com/support/ai/mcp/setup) first. ## Authentication methods The TestingBot MCP server supports three authentication methods: - **Browser login** , run the `tb_login` tool and authorize in a browser window on your local machine. No keys are ever pasted into your AI client. This is the recommended option for desktop use. - **Device-code flow** , a variant of `tb_login` for SSH, remote, headless or hosted web clients. You get a short code, open [https://testingbot.com/device](https://testingbot.com/device) on any device and authorize there. - **API key & secret** , set `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` in your client's `env` block. Best for CI and shared, non-interactive configurations. ### Configuration precedence When the server starts it resolves credentials in this fixed order: 1. **Environment variables** , if _both_ the key (`TESTINGBOT_KEY`) _and_ the secret (`TESTINGBOT_SECRET`) are set, they are used. 2. **Credentials file** , otherwise the server reads `~/.testingbot/credentials` (written by `tb_login`). 3. **Degraded mode** , if neither is available, every tool except `tb_login` replies that no credentials are configured. See [Degraded mode](https://testingbot.com#degraded) below. Environment variables always win. If you set a key and secret in your client's `env` block, the server uses those and never touches the credentials file, so you do not need to run `tb_login`. ## Browser login with tb\_login (loopback) The simplest way to authenticate on a desktop is the `tb_login` tool. It uses a loopback authorization flow (RFC 8252): the server opens your default browser to TestingBot, you click **Authorize** while signed in, and the credentials are captured on a local `127.0.0.1` callback. Nothing is ever copied, pasted or shown to your AI client. ### How it works, step by step 1. Ask your AI client to authenticate, for example: > "Log in to TestingBot" 1. Your client invokes the `tb_login` tool. 2. The server opens your default browser to TestingBot and starts a temporary local callback listener. 3. If you are not signed in yet, log in to your TestingBot account, then click **Authorize**. 4. The browser redirects back to the local `127.0.0.1` callback and the credentials are captured. 5. The server writes them to `~/.testingbot/credentials` and starts using them immediately, no restart needed. After this, all other tools (such as `getUserInfo`, `getTests` or `tb_openBrowser`) work automatically because the server reads the credentials file. Loopback login is the default on a desktop machine. You do not need to specify a `mode`, `auto` picks loopback when a browser is available. See [The mode parameter](https://testingbot.com#mode). Loopback login needs a local browser and the ability to open a `127.0.0.1` listener. On an SSH session, a container, a CI runner or a hosted web client there is no local browser to open, use the [device-code flow](https://testingbot.com#device-code) instead. ## Device-code flow When there is no local browser (SSH sessions, remote or headless hosts, containers, or web-based MCP clients), use the device-code flow (RFC 8628). Instead of opening a browser locally, the server prints a short code that you enter on any device with a browser. ### How it works, step by step 1. Run `tb_login` with the device mode, for example: > "Log in to TestingBot using the device code flow" 1. The server prints a short, human-readable code. 2. On any device, open [https://testingbot.com/device](https://testingbot.com/device). 3. Enter the code, sign in if needed, and click **Authorize**. 4. The server detects the approval, writes the credentials to `~/.testingbot/credentials` and starts using them. The device flow is **resumable**. If the polling window closes before you finish authorizing, just run `tb_login` again and the server will continue where it left off. You can tune the polling and lifetime windows with the [device/loopback environment variables](https://testingbot.com#env-vars). ## The mode parameter `tb_login` takes a single optional `mode` parameter that chooses the authorization flow. You can also set a default with the `TESTINGBOT_AUTH_MODE` environment variable. | Value | Best for | Behavior | | --- | --- | --- | | `auto` | Most users (default) | Picks `loopback` on a desktop and falls back to `device` on a headless host. | | `loopback` | Desktop machines | Opens the browser to TestingBot; you click Authorize; credentials are captured on a local `127.0.0.1` callback. Nothing pasted. | | `device` | SSH / remote / headless / web clients | Prints a short code; open [https://testingbot.com/device](https://testingbot.com/device), enter it and Authorize. Resumable. | For example, to force the device flow regardless of environment, ask your client to run `tb_login` with `mode` set to `device`, or set `TESTINGBOT_AUTH_MODE=device` in your client's `env` block. ## API key & secret As an alternative to `tb_login`, you can supply your TestingBot API key and secret directly. This is the preferred option for CI pipelines and shared configurations where there is no interactive browser. Add them to your MCP client's `env` block: ```json { "mcpServers": { "testingbot": { "command": "npx", "args": ["-y", "@testingbot/mcp-server@latest"], "env": { "TESTINGBOT_KEY": "", "TESTINGBOT_SECRET": "" } } } } ``` You can find your key and secret on your [TestingBot account page](https://testingbot.com/members/user/edit). Both must be set for the environment-variable path to take effect. Prefer `tb_login` for day-to-day local use, it keeps your key and secret out of config files entirely. Reach for the `env` block when you need a non-interactive setup such as CI. See [Setup & installation](https://testingbot.com/support/ai/mcp/setup) for the full per-client configuration. Treat your API secret like a password. Avoid committing config files that contain it to version control; use your CI provider's secret store and reference the values as environment variables instead. ## Where credentials live When you authenticate with `tb_login`, the server stores your credentials in an INI file at `~/.testingbot/credentials`. The file is organized into profile sections so you can keep more than one account on the same machine: ```bash [default] key = your-api-key secret = your-api-secret ``` By default the server reads the `[default]` profile. To use a different profile, set `TESTINGBOT_PROFILE`; to relocate the whole directory, set `TESTINGBOT_CONFIG_DIR`. ### File permissions The credentials are written with restrictive permissions so other users on the machine cannot read them: - The directory `~/.testingbot` is created with mode `0700` (owner only). - The `credentials` file is written with mode `0600` (owner read/write only). Credentials supplied through environment variables are never written to the credentials file. If you ever want to start fresh, delete `~/.testingbot/credentials` and run `tb_login` again. ## Environment variables The following environment variables can be set in your MCP client's `env` block (or your shell) to control authentication and connection behavior. Several have aliases for compatibility with existing TestingBot tooling. | Variable | Aliases | Default | Description | | --- | --- | --- | --- | | `TESTINGBOT_KEY` | `TB_KEY`, `TESTINGBOT_USERNAME` | – | Your TestingBot API key. | | `TESTINGBOT_SECRET` | `TB_SECRET`, `TESTINGBOT_ACCESS_KEY` | – | Your TestingBot API secret. Both key and secret must be set for the env path to apply. | | `TESTINGBOT_AUTH_MODE` | – | `auto` | Default `tb_login` mode: `auto`, `loopback` or `device`. | | `TESTINGBOT_BASE_URL` | – | `https://testingbot.com` | Override the TestingBot API endpoint. | | `TESTINGBOT_CONFIG_DIR` | – | `~/.testingbot` | Override the directory that holds the credentials file. | | `TESTINGBOT_PROFILE` | – | `default` | Credentials profile (section) name to read from the credentials file. | | `WDIO_LOG_LEVEL` | – | `silent` | WebDriver client log level. The server sets it to `silent` by default; rarely changed. | | `TESTINGBOT_DEVICE_POLL_WINDOW_MS` | – | `25000` | Device-code polling window, in milliseconds. | | `TESTINGBOT_LOOPBACK_WAIT_MS` | – | `60000` | How long the loopback flow waits for you to authorize, in milliseconds. | | `TESTINGBOT_LOOPBACK_LIFETIME_MS` | – | `900000` | Maximum lifetime of the loopback authorization session, in milliseconds. | ## Degraded mode If the server cannot find credentials (neither a key/secret pair in the environment nor a credentials file), it starts in **degraded mode**. In this state every tool except `tb_login` returns a message like: ```bash No TestingBot credentials configured. Run the tb_login tool to authenticate. ``` This is not an error in your client configuration, it simply means you have not authenticated yet. To fix it: - Run the `tb_login` tool (ask your client to "log in to TestingBot"), or - Set `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` in your client's `env` block and reload the server. Seeing the "Run tb\_login" message repeatedly after authenticating? Check the [Troubleshooting](https://testingbot.com/support/ai/mcp/troubleshooting) page, it is usually a profile or config-directory mismatch. ## Security notes The TestingBot MCP server is designed so your credentials stay under your control and never leak into your AI conversation. - **Credentials stay local.** Whether captured by `tb_login` or read from environment variables, they live on your machine (in `~/.testingbot/credentials` or the process environment) and are never sent to your LLM provider. - **Direct API calls.** The MCP server talks directly to the TestingBot API. Your AI client sends natural-language requests to the local server; the server performs the work and returns results. - **No key pasting.** The browser and device-code flows capture credentials out of band, so your key and secret never pass through the chat transcript. - **Restrictive permissions.** The credentials file is written `0600` inside a `0700` directory, readable only by your user account. Next steps: review the [Tool reference](https://testingbot.com/support/ai/mcp/tools) to see what you can do once authenticated, browse common [use cases & workflows](https://testingbot.com/support/ai/mcp/use-cases), or return to the [TestingBot MCP overview](https://testingbot.com/support/ai/mcp). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/ai/mcp/tools # Tool reference This page documents every tool exposed by the TestingBot MCP server. You never call these tools by hand, your AI client (Claude, Cursor, VS Code, Windsurf, and others) calls them on your behalf when you describe what you want in plain language. The names and parameters below are useful when you want to know exactly what an agent can do, or to phrase a prompt precisely. The tools come from two bundled packages. `@testingbot/mcp-server` handles your account and orchestration (tests, builds, storage, App Automate runs for Maestro, Espresso and XCUITest, screenshots, live sessions, team, tunnels, logs and project scaffolding). `@testingbot/automation-mcp`, which is bundled automatically, drives live browser and mobile sessions over WebDriver and proxies native app testing to Appium. See [Setup & installation](https://testingbot.com/support/ai/mcp/setup) to connect the server, and [Use cases & workflows](https://testingbot.com/support/ai/mcp/use-cases) for end-to-end examples that chain these tools together. Every tool below carries an example prompt. You do not have to phrase it that way, it is there to show the kind of request that resolves to that tool. Parameters are passed by your AI client, not typed by you. If a tool replies with _“No TestingBot credentials configured. Run the tb\_login tool…”_, authenticate first, see [Authentication](https://testingbot.com/support/ai/mcp/authentication). ## Authentication Logs you in without pasting an API key. After a successful login, credentials are written to `~/.testingbot/credentials` and used automatically, no restart needed. Read more on the [Authentication](https://testingbot.com/support/ai/mcp/authentication) page. | Tool | Description | | --- | --- | | `tb_login` | Browser or device-code login, no key paste required. | ### tb\_login | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `mode` | string | No | `auto` | Login flow: `auto` (loopback on a desktop, device on headless), `loopback` (RFC 8252 local callback), or `device` (RFC 8628 short code at [https://testingbot.com/device](https://testingbot.com/device)). | **Example prompt:** _“Log in to TestingBot”_ ## Browser & device Discover which browsers, platforms and mobile devices are available before you start a session. ### getBrowsers | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `type` | string | No | – | Filter the list: `web` or `mobile`. Omit for all. | **Example prompt:** _“Which Chrome versions can I run on Windows 11?”_ ### getDevices Lists mobile devices (real devices plus emulators/simulators) with their current availability. Takes no parameters. **Example prompt:** _“Which real iPhones are free right now?”_ ## Tests Browse, inspect and manage the test sessions in your account. ### getTests | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `offset` | number | No | `0` | Pagination offset. | | `limit` | number | No | `10` | Number of tests to return (max 100). | **Example prompt:** _“List my last 10 TestingBot tests”_ ### getTestDetails | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `sessionId` | string | Yes | – | The session to fetch full details for. | **Example prompt:** _“Show me the full details of my most recent failed test, with the video and logs”_ ### updateTest | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `sessionId` | string | Yes | – | The session to update. | | `name` | string | No | – | New test name. | | `status` | string | No | – | Mark the test as `passed` or `failed`. | | `build` | string | No | – | Associate the test with a build. | | `extra` | string | No | – | Additional metadata as a JSON string. | **Example prompt:** _“Mark that session as failed and rename it to Checkout regression”_ ### deleteTest | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `sessionId` | string | Yes | – | The session to delete. | **Example prompt:** _“Delete the session I just started by mistake”_ ### stopTest | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `sessionId` | string | Yes | – | The running session to stop. | **Example prompt:** _“Stop the session that is still running”_ ## Builds Group related tests into builds and manage them. ### getBuilds | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `offset` | number | No | `0` | Pagination offset. | | `limit` | number | No | `10` | Number of builds to return (max 100). | **Example prompt:** _“List my last 5 builds”_ ### getTestsForBuild | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `buildId` | string | Yes | – | The build whose tests you want to list. | **Example prompt:** _“Which tests are in the build called nightly?”_ ### deleteBuild | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `buildId` | string | Yes | – | Deletes the build and all of its tests. | **Example prompt:** _“Delete the build called scratch-run and everything in it”_ ## Storage Upload and manage native mobile app binaries (`.apk`, `.ipa`, `.zip`) so they can be installed on devices for native app testing. ### uploadFile | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `localFilePath` | string | Yes | – | Absolute path to a local `.apk`/`.ipa`/`.zip`. | **Example prompt:** _“Upload /Users/me/builds/app-release.apk to TestingBot storage”_ ### uploadRemoteFile | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `remoteUrl` | string | Yes | – | An http/https URL to fetch and store (SSRF-guarded). | **Example prompt:** _“Upload the build at https://ci.example.com/artifacts/app.ipa to my TestingBot storage”_ ### getStorageFiles | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `offset` | number | No | `0` | Pagination offset. | | `limit` | number | No | `10` | Number of files to return (max 100). | **Example prompt:** _“Which app builds do I have in storage?”_ ### deleteStorageFile | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `appUrl` | string | Yes | – | The stored app reference to delete. | **Example prompt:** _“Delete tb://app123 from storage”_ ## App Automate: Maestro Run [Maestro](https://testingbot.com/support/app-automate/maestro) flows on TestingBot real devices, emulators and simulators. The agent uploads your app, uploads (or writes) the flow YAML, starts a run, and polls until it finishes. Maestro runs are **asynchronous** : `runMaestroTest` returns a run ID immediately and does not block while devices execute. The agent then polls `getMaestroRunStatus` and fetches `getMaestroRunResults` once the run completes. | Tool | Description | | --- | --- | | `uploadMaestroApp` | Upload the app under test. Returns a project id. | | `uploadMaestroFlows` | Upload the flow YAML for a project, from disk or inline. | | `runMaestroTest` | Start a run. Returns a run id immediately. | | `getMaestroRunStatus` | Poll per-flow progress. | | `getMaestroRunResults` | Final outcomes plus the JUnit XML report. | | `getMaestroFlowDetails` | Errors, step-level report, video, screenshots and device logs for one flow. | | `cancelMaestroRun` | Cancel a running run. | | `retryMaestroRun` | Retry a whole run, or a single failed flow. | | `uploadMaestroCompanionApp` | Upload a companion app to install alongside the app under test. | | `listMaestroProjects` | Find existing projects instead of re-uploading. | | `listMaestroRuns` | List runs, or look them up by run name. | | `maestroCheatSheet` | Maestro YAML syntax reference for the agent. | ### uploadMaestroApp | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `localFilePath` | string | Yes | – | Local path to the `.apk`, `.aab`, `.ipa` or zipped `.app`. | Returns a **project id** that every other Maestro tool takes. Uploads are deduplicated by checksum, so re-running with an unchanged binary reuses the existing project instead of uploading again. **Example prompt:** _“Upload ./build/app.apk for Maestro testing”_ ### uploadMaestroFlows | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `projectId` | number | Yes | – | Project id from `uploadMaestroApp`. | | `flowsPath` | string | No | – | Directory of `.yaml`/`.yml` flows (structure preserved, so `runFlow` subflows keep resolving), or a pre-built `.zip`. | | `flows` | array | No | – | Inline flows as `{ fileName, content }` objects, for flows the agent writes itself. | Each upload replaces the flows previously uploaded for that project. **Example prompt:** _“Upload the flows in ./flows to that project”_ ### runMaestroTest | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `projectId` | number | Yes | – | Project id from `uploadMaestroApp`. | | `platformName` | string | Yes | – | `Android` or `iOS`. | | `deviceName` | string | No | any device | Device name, wildcards and regex supported, e.g. `Pixel 8`, `iPhone 1[4-6]`, `*`. | | `version` | string | No | – | OS version, e.g. `14`. | | `realDevice` | boolean | No | `false` | Run on a physical device instead of an emulator/simulator. Required for `.ipa` apps. | | `name` | string | No | – | Run name shown in the dashboard, and the value `listMaestroRuns` looks up. | | `env` | object | No | – | Environment variables passed to the flows. | | `includeTags` | array | No | – | Only run flows carrying these Maestro tags. | | `excludeTags` | array | No | – | Skip flows carrying these Maestro tags. | | `shardSplit` | number | No | – | Split the flows across this many parallel devices (1–10). | | `otherApps` | array | No | – | Up to 4 companion app URLs (`tb://` or `http(s)://`) installed alongside the app under test. | **Example prompt:** _“Run my Maestro flows on a real Pixel 8 with Android 14”_ ### getMaestroRunStatus | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `projectId` | number | Yes | – | The project to inspect. | | `runId` | number | No | – | A specific run. Omit it to list the runs of the latest execution. | Per-flow statuses are `WAITING`, `READY` (running), `DONE`, `FAILED` and `CANCELLED`, with failures listed first. The flow ids in this output are what `getMaestroFlowDetails` and `retryMaestroRun` take. **Example prompt:** _“How is that Maestro run doing?”_ ### getMaestroRunResults | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `projectId` | number | Yes | – | The project. | | `runId` | number | Yes | – | The completed run. | Returns per-flow outcomes, failures first with error messages and video links, plus the JUnit XML report. **Example prompt:** _“Give me the results of the Maestro run now that it has finished”_ ### getMaestroFlowDetails | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `projectId` | number | Yes | – | The project. | | `runId` | number | Yes | – | The run. | | `flowId` | number | Yes | – | The flow, taken from `getMaestroRunStatus`. | This is the debugging tool: error messages, the step-level JUnit report and links to the video, screenshots and device logs. Fetch it for a failed flow, fix the YAML, upload the flows again, then retry. **Example prompt:** _“Why did the checkout flow fail? Show me the error and the video”_ ### cancelMaestroRun | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `projectId` | number | Yes | – | The project. | | `runId` | number | Yes | – | The run to cancel. | **Example prompt:** _“Cancel the Maestro run I just started”_ ### retryMaestroRun | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `projectId` | number | Yes | – | The project. | | `runId` | number | Yes | – | The run to retry. | | `flowId` | number | No | – | Retry only this flow instead of the whole run. | **Example prompt:** _“Fix the login flow and re-run just that one”_ ### uploadMaestroCompanionApp | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `localFilePath` | string | Yes | – | Local path to the companion `.apk`, `.aab`, `.ipa` or `.zip`. | Returns a `tb://` app URL to pass in the `otherApps` parameter of `runMaestroTest` (max 4 per run). **Example prompt:** _“Upload ./build/mock-backend.apk as a companion app for the run”_ ### listMaestroProjects | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `offset` | number | No | `0` | Pagination offset. | | `count` | number | No | `10` | Number of projects to return (max 100). | Newest first, with app details, flow names and run ids, so the agent can reuse a project id rather than re-uploading an app. **Example prompt:** _“What Maestro projects do I already have on my account?”_ ### listMaestroRuns | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `buildName` | string | No | – | Look up runs with this exact run name instead of listing all runs. | | `offset` | number | No | `0` | Pagination offset. Ignored when `buildName` is set. | | `count` | number | No | `10` | Number of runs to return (max 100). Ignored when `buildName` is set. | Returns `projectId` and `runId` pairs. The `buildName` lookup is what makes prompts like _“rerun whatever failed in last night's nightly build”_ work. **Example prompt:** _“Find the Maestro run named nightly and tell me what failed in it”_ ### maestroCheatSheet Returns a Maestro YAML syntax reference: file structure, commands, selectors and the TestingBot run workflow. Takes no parameters. Agents call this before writing or fixing flows, so the YAML they produce matches what Maestro actually accepts. **Example prompt:** _“Write me a Maestro flow that logs in and adds an item to the cart”_ Maestro tooling requires `@testingbot/mcp-server` 1.1.0 or later. If your agent reports that these tools do not exist, update the server, see [Setup & installation](https://testingbot.com/support/ai/mcp/setup). ## App Automate: Espresso & XCUITest Run pre-built [Espresso](https://testingbot.com/support/app-automate/espresso) (Android) and [XCUITest](https://testingbot.com/support/app-automate/xcuitest) (iOS) suites on TestingBot devices. Every tool takes a `framework` parameter, `espresso` or `xcuitest`, and follows the same asynchronous start, poll, fetch pattern as Maestro. | Tool | Description | | --- | --- | | `uploadAppAutomateApp` | Upload the app under test. Returns a project id. | | `uploadAppAutomateTests` | Upload the test suite for that project. | | `runAppAutomateTest` | Start a run. Returns a run id immediately. | | `getAppAutomateRunStatus` | Poll run progress. | | `getAppAutomateRunResults` | Per-run outcomes plus the JUnit XML report. | ### uploadAppAutomateApp | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `framework` | string | Yes | – | `espresso` or `xcuitest`. | | `localFilePath` | string | Yes | – | The app under test: `.apk`/`.aab` for Espresso, `.ipa`/`.zip` for XCUITest. | Returns the project id used by the other two App Automate tools. For Maestro flows, use `uploadMaestroApp` instead. **Example prompt:** _“Upload ./app/build/outputs/apk/debug/app-debug.apk for an Espresso run”_ ### uploadAppAutomateTests | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `framework` | string | Yes | – | `espresso` or `xcuitest`. | | `projectId` | number | Yes | – | Project id from `uploadAppAutomateApp`. | | `localFilePath` | string | Yes | – | The instrumented test `.apk` for Espresso, or the zipped XCUITest runner bundle (`.zip`). | **Example prompt:** _“Upload the instrumented test APK for that project”_ ### runAppAutomateTest | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `framework` | string | Yes | – | `espresso` or `xcuitest`. | | `projectId` | number | Yes | – | Project id from `uploadAppAutomateApp`. | | `deviceName` | string | No | any device | Device name, wildcards and regex supported, e.g. `Galaxy S23`, `iPhone 1[4-6]`, `*`. | | `version` | string | No | – | OS version, e.g. `14`. | | `realDevice` | boolean | No | `false` | Run on a physical device instead of an emulator/simulator. | | `name` | string | No | – | Run name shown in the dashboard. | | `build` | string | No | – | Build identifier used to group runs. | | `locale` | string | No | – | Locale for the run, e.g. `fr_FR`. | | `timeZone` | string | No | – | Time zone for the run, e.g. `Europe/Brussels`. | | `testClasses` | array | No | – | Espresso only: run only these fully qualified test classes. | | `skipTestClasses` | array | No | – | Espresso only: skip these test classes. | | `packages` | array | No | – | Espresso only: run only these packages. | | `annotations` | array | No | – | Espresso only: run only tests carrying these annotations. | | `testRunner` | string | No | – | Espresso only: custom instrumentation runner. | **Example prompt:** _“Run the Espresso suite on a real Galaxy S23, only the com.example.LoginTest class”_ ### getAppAutomateRunStatus | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `framework` | string | Yes | – | `espresso` or `xcuitest`. | | `projectId` | number | Yes | – | The project to inspect. | | `runId` | number | No | – | A specific run. Omit it to list every run in the project. | Statuses are `WAITING`, `READY` (running), `DONE` and `FAILED`. Each run also reports its Appium session id. **Example prompt:** _“Is my XCUITest run finished yet?”_ ### getAppAutomateRunResults | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `framework` | string | Yes | – | `espresso` or `xcuitest`. | | `projectId` | number | Yes | – | The completed project. | | `runId` | number | No | – | XCUITest only: fetch the JUnit report of this single run. | **Example prompt:** _“Show me the JUnit results for that Espresso project”_ Because each Espresso and XCUITest run surfaces its Appium session id, the session tools work on them too: pass that id to [`getTestDetails`](https://testingbot.com#tests) for the video and screenshots, or to [`getFailureLogs`](https://testingbot.com#logs) for the device and Appium logs. ## Screenshots Capture a URL across multiple browsers and platforms for cross-browser visual checks. ### takeScreenshot | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `url` | string | Yes | – | The page to capture. | | `browsers` | array | Yes | – | Array of `{ browserName, version?, os }` targets. | | `resolution` | string | No | `1920x1080` | Screen resolution to render at. | | `waitTime` | number | No | `5` | Seconds to wait before capture (0–60). | | `fullPage` | boolean | No | `false` | Capture the full scrollable page. | Returns a job id that you pass to `retrieveScreenshots`. **Example prompt:** _“Take full-page screenshots of testingbot.com in Chrome on Windows 11 and Safari on macOS”_ ### retrieveScreenshots | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `screenshotId` | string | Yes | – | The job id returned by `takeScreenshot`. | **Example prompt:** _“Are those screenshots ready? Give me the image links”_ ### getScreenshotList | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `offset` | number | No | `0` | Pagination offset. | | `limit` | number | No | `10` | Number of jobs to return (max 100). | **Example prompt:** _“List my recent screenshot jobs”_ ## User Read and update your account profile. ### getUserInfo Returns account information, name, plan, company, country, available seconds and concurrency. Takes no parameters. Note: `getUserInfo` does not return an email field. **Example prompt:** _“How many testing minutes do I have left?”_ ### updateUserInfo | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `firstName` | string | No | – | New first name. | | `lastName` | string | No | – | New last name. | | `email` | string | No | – | New email address. | **Example prompt:** _“Update the last name on my profile to De Vries”_ ## Live testing Start an interactive, manual live session you can watch and control. Each session returns a live-view URL of the form `https://testingbot.com/tests//live?auth=`. ### startLiveSession | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `platformType` | string | Yes | – | `desktop` or `mobile`. | | `desiredURL` | string | Yes | – | The URL to open in the session. | | `desiredOS` | string | Yes | – | Operating system to run on. | | `desiredOSVersion` | string | Yes | – | OS version. | | `desiredBrowser` | string | No | – | Browser to launch (desktop). | | `desiredBrowserVersion` | string | No | – | Browser version (desktop). | | `desiredDevice` | string | No | – | Device name (mobile). | **Example prompt:** _“Start a live session on Windows 11 in Chrome and open my staging site”_ ### startDesktopLiveSession | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `desiredURL` | string | Yes | – | The URL to open. | | `desiredOS` | string | Yes | – | `Windows`, `Mac` or `Linux`. | | `desiredOSVersion` | string | Yes | – | OS version. | | `desiredBrowser` | string | Yes | – | `chrome`, `firefox`, `safari`, `edge` or `ie`. | | `desiredBrowserVersion` | string | No | `latest` | Browser version. | **Example prompt:** _“Give me a manual Safari session on macOS Ventura pointed at example.com”_ ### startMobileLiveSession | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `desiredURL` | string | Yes | – | The URL to open. | | `desiredOS` | string | Yes | – | `android` or `ios`. | | `desiredOSVersion` | string | Yes | – | OS version. | | `desiredDevice` | string | Yes | – | The device to run on. | **Example prompt:** _“Open a live session on an iPhone 15 Pro and load my checkout page”_ ## Team Inspect your team settings, usage and members. | Tool | Param | Required | Description | | --- | --- | --- | --- | | `getTeam` | – | – | Team settings and usage. No parameters. | | `getUsersInTeam` | – | – | List all team members. No parameters. | | `getUserFromTeam` | `userId` | Yes | Fetch a single team member by id. | **Example prompt:** _“Show my team's usage this month and list everyone in it”_ ## CDP Spin up a remote browser exposing a Chrome DevTools Protocol endpoint, so you can drive it with Puppeteer or Playwright. See the Playwright/Puppeteer example in [Use cases & workflows](https://testingbot.com/support/ai/mcp/use-cases). ### createCdpSession | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `browserName` | string | Yes | – | Browser to launch. | | `browserVersion` | string | No | `latest` | Browser version. | | `platform` | string | Yes | – | Platform to run on. | | `screenResolution` | string | No | – | Screen resolution. | | `timeZone` | string | No | – | Time zone for the session. | | `name` | string | No | – | Session name. | | `build` | string | No | – | Build to associate the session with. | | `extraCapabilities` | object | No | – | Additional WebDriver capabilities. | **Example prompt:** _“Start a CDP-enabled Chrome session on Windows 11 and give me the WebSocket URL”_ ## Tunnels Manage TestingBot Tunnels for testing local or firewalled sites. | Tool | Param | Required | Description | | --- | --- | --- | --- | | `getTunnelList` | – | – | List active tunnels. No parameters. | | `deleteTunnel` | `tunnelId` | Yes | Stop an active tunnel by id. | **Example prompt:** _“List my active tunnels and stop the one I left running this morning”_ ## Logs Fetch and filter the logs for a session, ideal for debugging a failing run. ### getFailureLogs | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `sessionId` | string | Yes | – | The session whose logs you want. | | `logTypes` | array | No | – | Any of `selenium`, `browser`, `chrome`, `vm`, `appium`. | | `failuresOnly` | boolean | No | `false` | Return only failure-related entries. | | `maxBytesPerLog` | number | No | `50000` | Truncate each log (1,000–1,000,000 bytes). | | `timeoutMs` | number | No | `15000` | Fetch timeout (1,000–60,000 ms). | **Example prompt:** _“Pull just the failure lines from the Appium and device logs for that session”_ ## Project Detect the test framework in a local project and scaffold a ready-to-use TestingBot configuration. ### listTestFiles | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `projectRoot` | string | Yes | – | Root directory of the project to scan. | | `maxDepth` | number | No | `8` | How deep to recurse (1–15). | | `maxResults` | number | No | `200` | Maximum files to return (1–2000). | **Example prompt:** _“What test files are in this project?”_ ### setupTestingBot | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `projectRoot` | string | Yes | – | Root directory of the project. | | `frameworkOverride` | string | No | – | Force a framework instead of auto-detecting. | Generates a ready-to-paste TestingBot config for the detected framework, Playwright, Cypress, WebdriverIO, Selenium (JS), Nightwatch or Puppeteer. **Example prompt:** _“Set this project up to run on TestingBot”_ ## Browser automation These tools come from the bundled `@testingbot/automation-mcp` package and drive a live browser session over WebDriver. They work for desktop browsers and for mobile browsers (on emulators/simulators or real devices). Every session starts with `tb_openBrowser`, which returns a `sessionId` you pass to the other tools, plus a live-view URL (`https://testingbot.com/tests//live?auth=`) you can open to watch or take over. ### tb\_openBrowser | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `browserName` | string | Yes | – | `chrome`, `firefox`, `edge` or `safari`. | | `browserVersion` | string | No | `latest` | Browser version (desktop only). | | `platform` | string | Yes | – | Desktop codes like `WIN11`/`VENTURA`, or `Android`/`iOS` for mobile. | | `deviceName` | string | No | – | Mobile only (e.g. `Google Pixel 9`, `iPhone 15 Pro`). Its presence switches the session to mobile. | | `platformVersion` | string | No | – | Mobile OS version. | | `automationName` | string | No | `UiAutomator2`/`XCUITest` | Mobile automation engine. | | `realDevice` | boolean | No | `false` | `true` = physical device, `false` = emulator/simulator. | | `name` | string | No | – | Session name. | | `build` | string | No | – | Build to group the session under. | | `screenResolution` | string | No | – | Screen resolution (desktop only). | **Example prompt:** _“Open Chrome on Windows 11”_ Real-device pre-flight: if `realDevice:true` and the requested device or version is busy or not offered, `tb_openBrowser` returns guidance (wait, use an emulator, or pick an available alternative) **without** starting a session. See [Use cases](https://testingbot.com/support/ai/mcp/use-cases) and [Troubleshooting](https://testingbot.com/support/ai/mcp/troubleshooting). ### tb\_navigate | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `sessionId` | string | Yes | – | The active session. | | `url` | string | Yes | – | The URL to navigate to. | **Example prompt:** _“Go to my staging login page”_ ### tb\_snapshot | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `sessionId` | string | Yes | – | The active session. | | `bodyChars` | number | No | `5000` | Body text characters to include (500–50,000). | Returns a structured summary of the page: title, headings, actionable elements and body text. **Example prompt:** _“What is on the page right now?”_ ### tb\_click | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `sessionId` | string | Yes | – | The active session. | | `by` | string | No | `css` | Locator strategy: `css`, `xpath`, `id`, `name`, `tag`, `class`, `linkText`, `partialLinkText`. | | `value` | string | Yes | – | The locator value. | | `timeoutMs` | number | No | `5000` | Wait timeout (100–30,000 ms). | **Example prompt:** _“Click the Sign in button”_ ### tb\_type | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `sessionId` | string | Yes | – | The active session. | | `by` | string | No | `css` | Locator strategy. | | `value` | string | Yes | – | The locator value of the field. | | `text` | string | Yes | – | Text to type. | | `clearFirst` | boolean | No | `true` | Clear the field before typing. | | `pressEnter` | boolean | No | `false` | Press Enter after typing. | **Example prompt:** _“Type test@example.com into the email field and press Enter”_ ### tb\_getText | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `sessionId` | string | Yes | – | The active session. | | `by` | string | No | `css` | Locator strategy. | | `value` | string | Yes | – | The locator value. | | `timeoutMs` | number | No | `5000` | Wait timeout. | **Example prompt:** _“Read back the dashboard heading”_ ### tb\_getAttribute | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `sessionId` | string | Yes | – | The active session. | | `by` | string | No | `css` | Locator strategy. | | `value` | string | Yes | – | The locator value. | | `attribute` | string | Yes | – | The attribute name to read. | | `timeoutMs` | number | No | `5000` | Wait timeout. | **Example prompt:** _“Is the submit button still disabled?”_ ### tb\_executeScript | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `sessionId` | string | Yes | – | The active session. | | `script` | string | Yes | – | JavaScript to run in the page. | | `args` | array | No | `[]` | Arguments passed to the script. | **Example prompt:** _“Scroll to the bottom of the page and tell me the document height”_ ### tb\_screenshot | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `sessionId` | string | Yes | – | The active session. | | `savePath` | string | No | OS temp dir | A `.png` path or directory to save to. | Returns an inline image _and_ saves a file, returning a clickable `file://` link. Many clients do not render inline tool images, so open the saved file if you do not see the screenshot in chat. **Example prompt:** _“Take a screenshot of what the browser is showing now”_ ### tb\_pressKey | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `sessionId` | string | Yes | – | The active session. | | `key` | string | Yes | – | A W3C key or chord, e.g. `Enter` or `Control+A`. | **Example prompt:** _“Press Escape to close the modal”_ ### tb\_closeBrowser | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `sessionId` | string | Yes | – | Ends the session and frees minutes. | **Example prompt:** _“Close the session”_ ### tb\_listSessions Lists active automation sessions. Takes no parameters. **Example prompt:** _“Which automation sessions do I have open?”_ ## Mobile native apps For testing native iOS/Android apps (not mobile browsers, use `tb_openBrowser` for those), the automation package proxies to the official appium-mcp. Upload your app first with `uploadFile`, then reference it as `appium:app="tb://"`. ### appium\_session\_management | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `action` | string | Yes | – | One of `create`, `delete`, `list`, `detach`, `select`. | | `platform` | string | For create | – | `ios` or `android`. | | `capabilities` | string | No | – | JSON string for `create`: `appium:deviceName`, `appium:platformVersion`, `appium:app="tb://"`. | | `sessionId` | string | No | – | Required for `delete`, `detach` and `select`. | Ready to put these together? See [Use cases & workflows](https://testingbot.com/support/ai/mcp/use-cases) for full tool sequences, or return to the [MCP overview](https://testingbot.com/support/ai/mcp). **Example prompt:** _“Start a native Android session with the app I just uploaded”_ ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/ai/mcp/use-cases # Use cases & workflows TestingBot MCP turns natural language into real testing work. You describe a goal to your AI client and the agent decides which TestingBot tools to call, in what order, to get it done. This page walks through the most common workflows: each scenario lists the goal, an example prompt you could give your agent, and the sequence of tools the agent would typically run behind the scenes. You never call these tools by hand, your client does. For the full list of tools and their parameters, see the [Tool reference](https://testingbot.com/support/ai/mcp/tools). If you have not set the server up yet, start with [Setup & installation](https://testingbot.com/support/ai/mcp/setup) and [Authentication](https://testingbot.com/support/ai/mcp/authentication). All of these workflows run on TestingBot's cloud grid through one locally-running server. The main `@testingbot/mcp-server` package handles your account, tests, builds, storage and screenshots; it bundles `@testingbot/automation-mcp`, which drives live browser and mobile sessions. You don't install browsers, emulators or Appium locally. See [the MCP overview](https://testingbot.com/support/ai/mcp) for the big picture. ## 1. Automate a real desktop browser **Goal:** open a real desktop browser on TestingBot's cloud, navigate to a page and interact with it (click, type, read text and assert), all from natural language. > "Open Chrome on Windows 11, go to my staging login page, sign in with test@example.com / hunter2, and tell me whether the dashboard heading appears." A typical tool sequence the agent runs: 1. `tb_openBrowser`: start a Chrome session on `WIN11`; returns a `sessionId` and a live-view URL. 2. `tb_navigate`: load the staging login page. 3. `tb_snapshot`: get a structured summary of the page so the agent knows what to interact with. 4. `tb_type`: fill the email and password fields. 5. `tb_click`: submit the form. 6. `tb_getText`: read the dashboard heading to confirm the login worked. 7. `tb_closeBrowser`: end the session and free your minutes. The same browser-automation tools work for desktop and mobile browsers, the difference is the capabilities passed to `tb_openBrowser`. You can watch the whole run live (see [Watch or take over a session](https://testingbot.com#live-view)). ## 2. Mobile browser on an emulator/simulator **Goal:** test your mobile web experience on Android or iOS without owning the hardware. Emulators and simulators are the default, they are cheaper and plentiful, which makes them ideal for everyday responsive and functional checks. > "Open Chrome on an Android emulator (Google Pixel 9), load testingbot.com and check that the mobile menu button is visible." A typical tool sequence: 1. `getDevices`: (optional) list the mobile devices that are available. 2. `tb_openBrowser`: with `platform` `Android`, `deviceName` `"Google Pixel 9"` and `realDevice: false` (the default). Providing a `deviceName` switches the session to mobile. 3. `tb_navigate`: load the URL. 4. `tb_snapshot` / `tb_getAttribute`: inspect the page and confirm the menu button is present. 5. `tb_closeBrowser`: end the session. Leaving `realDevice` at its default (`false`) runs on an emulator (Android) or simulator (iOS). This is the right choice for most mobile-web testing. Browse the physical device list at [/support/app-automate/devices](https://testingbot.com/support/app-automate/devices). ## 3. Mobile browser on a physical real device **Goal:** run your mobile web test on real hardware when you need true GPU rendering, sensors, or the genuine mobile Chrome/Safari engine, not an emulator approximation. > "Open Safari on a real iPhone 15 Pro running iOS 17, go to my checkout flow and read back the total price element." A typical tool sequence: 1. `getDevices`: check which real devices and OS versions are currently free. 2. `tb_openBrowser`: with `platform` `iOS`, `deviceName` `"iPhone 15 Pro"`, `platformVersion` `"17"` and `realDevice: true`. 3. `tb_navigate`: load the checkout flow. 4. `tb_getText`: read the total price element. 5. `tb_closeBrowser`: end the session. Real devices are a shared, finite resource. When `realDevice: true` and the requested device or OS version is busy or not offered, `tb_openBrowser` runs a pre-flight check and returns guidance _without_ starting (and billing) a session. It will suggest waiting for the device to free up, picking an available alternative device/version, or falling back to an emulator/simulator. Your agent can act on that guidance and retry. ## 4. Native mobile app testing **Goal:** test a native Android (`.apk`) or iOS (`.ipa`) app, not a mobile website. Upload the build to TestingBot storage, then start an Appium session against it. > "Upload the app at /Users/me/builds/app-release.apk to TestingBot, then start a native Android session with it." A typical tool sequence: 1. `uploadFile`: upload the local `.apk`/`.ipa`/`.zip` by absolute path. (Use `uploadRemoteFile` if the build lives at a URL.) This returns an app URL like `tb://`. 2. `getStorageFiles`: (optional) confirm the upload and look up the app URL. 3. `appium_session_management`: `action: "create"`, `platform: "android"`, with `capabilities` including `appium:app="tb://"`, `appium:deviceName` and `appium:platformVersion`. 4. `appium_session_management`: `action: "list"` / `"select"` / `"delete"` to manage and end sessions. Native app sessions are managed by `appium_session_management` (proxied to the official Appium MCP). For mobile _browser_ testing, use `tb_openBrowser` with a `deviceName` instead, see scenarios 2 and 3 above. To run an existing test suite rather than drive the app step by step, use App Automate, see scenarios 5 and 6 below. ## 5. Run a Maestro flow suite and fix what fails **Goal:** run your [Maestro](https://testingbot.com/support/app-automate/maestro) flows against a build on real devices, emulators or simulators, then have the agent read the failures, correct the YAML and re-run only what broke. > "Run the Maestro flows in ./flows against ./build/app.apk on a real Pixel 8 with Android 14. Fix any failures and re-run them." A typical tool sequence: 1. `uploadMaestroApp`: upload the `.apk`/`.aab`/`.ipa`. Returns a `projectId`. An unchanged binary is deduplicated by checksum, so repeat runs skip the upload. 2. `maestroCheatSheet`: (when the agent is writing or repairing flows) fetch the YAML syntax reference first. 3. `uploadMaestroFlows`: upload the flow directory, a `.zip`, or inline YAML the agent wrote itself. 4. `runMaestroTest`: start the run with `platformName: "Android"`, `deviceName: "Pixel 8"`, `version: "14"` and `realDevice: true`. Returns a `runId` immediately. 5. `getMaestroRunStatus`: poll until every flow reports `DONE`, `FAILED` or `CANCELLED`. 6. `getMaestroRunResults`: per-flow outcomes plus the JUnit XML report. 7. `getMaestroFlowDetails`: for a failed flow, read the error, the step-level report, and the video, screenshot and device-log links. 8. `uploadMaestroFlows` then `retryMaestroRun`: push the corrected YAML and retry, passing `flowId` to re-run just that one flow. Already have a project on the account? `listMaestroProjects` and `listMaestroRuns` let the agent find it without re-uploading anything. `listMaestroRuns` also takes a `buildName`, which is what makes a prompt like _"rerun whatever failed in last night's nightly build"_ resolve to a concrete run. Useful extras on `runMaestroTest`: `includeTags`/`excludeTags` to run a subset (a smoke tag, say), `shardSplit` to spread the flows over parallel devices, `env` to inject credentials or feature flags, and `otherApps` (with `uploadMaestroCompanionApp`) to install up to 4 companion apps alongside the app under test. `cancelMaestroRun` stops a run you no longer need. ## 6. Run an Espresso or XCUITest suite **Goal:** run a pre-built [Espresso](https://testingbot.com/support/app-automate/espresso) (Android) or [XCUITest](https://testingbot.com/support/app-automate/xcuitest) (iOS) suite on TestingBot devices and get the JUnit report back, without writing a CI job for it. > "Run my Espresso suite on a real Galaxy S23, but only the com.example.LoginTest class, and show me the failures." A typical tool sequence: 1. `uploadAppAutomateApp`: `framework: "espresso"` with the app `.apk`/`.aab` (or `"xcuitest"` with the `.ipa`). Returns a `projectId`. 2. `uploadAppAutomateTests`: the instrumented test `.apk` for Espresso, or the zipped XCUITest runner bundle. 3. `runAppAutomateTest`: start the run with `deviceName`, `version` and `realDevice`. For Espresso, narrow the scope with `testClasses`, `skipTestClasses`, `packages` or `annotations`. 4. `getAppAutomateRunStatus`: poll until the runs finish. 5. `getAppAutomateRunResults`: per-run outcomes plus the JUnit XML report. 6. `getTestDetails` / `getFailureLogs`: each run reports an Appium session id, so the usual session tools give you the video, screenshots and device logs for a failure. `locale` and `timeZone` set the device environment for the run, and `build` groups related runs together in the dashboard. ## 7. Cross-browser screenshots **Goal:** capture how a single URL renders across many browsers and operating systems at once for fast visual comparison, without opening interactive sessions. > "Take full-page screenshots of testingbot.com in Chrome on Windows 11 and Safari on macOS, then give me the image links." A typical tool sequence: 1. `getBrowsers`: (optional) confirm valid `browserName`/`version`/`os` combinations. 2. `takeScreenshot`: pass the `url` and a `browsers` array (each entry has `browserName`, optional `version` and `os`). Optionally set `resolution`, `waitTime` and `fullPage`. Returns a job id. 3. `retrieveScreenshots`: pass the `screenshotId` (the job id) to get the captured image URLs. 4. `getScreenshotList`: (optional) list earlier screenshot jobs. This is a batch, fire-and-collect flow: `takeScreenshot` queues the job and `retrieveScreenshots` fetches the results once they are ready. ## 8. Drive a session with Playwright/Puppeteer via CDP **Goal:** get a remote cloud browser plus a Chrome DevTools Protocol (CDP) endpoint so your own Playwright or Puppeteer scripts can connect to a TestingBot browser. > "Start a CDP-enabled Chrome session on Windows 11 and give me the CDP URL so I can connect Playwright to it." A typical tool sequence: 1. `createCdpSession`: pass `browserName` and `platform` (and optionally `browserVersion`, `screenResolution`, `timeZone`, `name`, `build`, `extraCapabilities`). Returns a remote browser and a CDP URL. 2. Connect your Playwright/Puppeteer code to the returned CDP URL and run your script. 3. `getTests` / `getTestDetails`: review the run afterwards on TestingBot. Use this when you already have framework code and just want it pointed at the TestingBot grid, rather than driving the browser through the `tb_*` automation tools. ## 9. Watch or take over a session **Goal:** watch an automation session in real time, or step in and drive it manually when something needs a human. > "Open a Chrome session on Windows 11 and give me the live link so I can watch what the agent is doing." How it works: 1. `tb_openBrowser` (or any session-starting tool) returns a live-view URL of the form `https://testingbot.com/tests//live?auth=`. 2. Open that URL in your browser to watch the session live; you can interact with it directly to take over. 3. `tb_listSessions`: list active automation sessions if you lose track of which is running. 4. `stopTest` or `tb_closeBrowser`, stop the session when you are done. Every automation session returns its own live-view URL, so you can monitor or intervene at any point. This is useful for one-time captchas, MFA prompts, or simply seeing the agent's actions as they happen. ## 10. Inspect & debug a run **Goal:** figure out why a test failed by pulling its details and logs after the fact. > "Show me my last 5 tests, then pull the details and failure logs for the one that failed." A typical tool sequence: 1. `getTests`: list recent tests (paginated via `offset`/`limit`) to find the session in question. 2. `getTestDetails`: full detail for a single session by `sessionId`. 3. `getFailureLogs`: fetch and filter session logs by `sessionId`; narrow with `logTypes` (selenium, browser, chrome, vm, appium) and `failuresOnly: true`. 4. `updateTest`: (optional) mark the test `passed`/`failed` or update its name/build once you know the outcome. Working with builds instead of individual tests? Use `getBuilds` and `getTestsForBuild` to drill from a build down to its sessions, then debug each one the same way. For the complete catalog of tools and every parameter referenced above, see the [Tool reference](https://testingbot.com/support/ai/mcp/tools). If a workflow misbehaves, the [Troubleshooting](https://testingbot.com/support/ai/mcp/troubleshooting) guide covers the most common issues, and the [FAQ](https://testingbot.com/support/ai/mcp/faq) answers the rest. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/ai/mcp/troubleshooting # Troubleshooting This page collects the issues we see most often when running the TestingBot MCP server, in an **Issue → Cause → Fix** format. If you have not configured the server yet, start with [Setup & installation](https://testingbot.com/support/ai/mcp/setup); for anything related to logging in, see [Authentication](https://testingbot.com/support/ai/mcp/authentication). Most problems fall into one of two buckets: the server can't authenticate (so tools refuse to run), or a requested resource (a real device, a specific OS version) isn't available right now. Both are handled gracefully by the server, it returns guidance rather than failing silently. When something goes wrong, the quickest diagnostic is to ask your agent to run `getUserInfo`. If it returns your account details, authentication is fine and the problem is elsewhere. If it replies "No TestingBot credentials configured", jump to [the credentials section](https://testingbot.com#credentials) below. ## Tools reply "Run tb\_login" or "No credentials" ### Issue Every tool except `tb_login` replies with something like: ```bash No TestingBot credentials configured. Run the tb_login tool to authenticate. ``` ### Cause The server is running in **degraded mode**. It could not find usable credentials, so it disables every tool that talks to TestingBot and only keeps `tb_login` available. The server looks for credentials in this order: 1. **Environment variables** : both `TESTINGBOT_KEY` _and_ `TESTINGBOT_SECRET` must be set. Setting only one is not enough. 2. **Credentials file** : `~/.testingbot/credentials`, written automatically after a successful `tb_login`. 3. **Degraded mode** : neither of the above was found. ### Fix Pick whichever fits your environment: - **Interactive desktop:** ask your agent to run the `tb_login` tool. A browser opens, you click Authorize, and credentials are captured automatically, with nothing pasted. No restart is needed; the next tool call will work. - **SSH / remote / headless:** run `tb_login` with `mode` set to `device` (see [below](https://testingbot.com#headless-login)). - **CI or a shared config:** set the key and secret in your client's `env` block. Generate them at [testingbot.com/members/user/edit](https://testingbot.com/members/user/edit). ```json { "mcpServers": { "testingbot": { "command": "npx", "args": ["-y", "@testingbot/mcp-server@latest"], "env": { "TESTINGBOT_KEY": "", "TESTINGBOT_SECRET": "" } } } } ``` If you set only `TESTINGBOT_KEY` or only `TESTINGBOT_SECRET`, the env layer is ignored entirely and the server falls through to the credentials file (or degraded mode). Always set both, or use `tb_login` instead. For the full authentication model (loopback vs device flow, profiles, and where credentials are stored), see [Authentication](https://testingbot.com/support/ai/mcp/authentication). ## Requested real device is unavailable, busy, or the wrong OS version ### Issue You asked to automate a physical device (for example "open Safari on a real iPhone 15 Pro") and the agent comes back with guidance instead of a running session, it tells you the device is busy, the OS version isn't offered, or suggests an alternative. ### Cause When `tb_openBrowser` is called with `realDevice: true`, the server runs a **pre-flight check** against the real-device cloud before consuming any minutes. If the exact device/version combination is currently in use or simply not offered, it returns advice _without_ starting a session, so you don't burn time on a session that can't start. ### Fix - **Wait and retry** : physical devices are shared. If the device is just busy, try again shortly. - **Pick an offered version** : ask the agent to list devices first (`getDevices`) and choose an available OS version. You can also browse the physical device list at [/support/app-automate/devices](https://testingbot.com/support/app-automate/devices). - **Use an emulator or simulator instead** : drop `realDevice` (it defaults to `false`) to run the same mobile browser on an emulator/simulator. These are cheaper and far more plentiful, and are usually fine unless you specifically need real hardware (GPU, sensors, real Chrome/Safari). Try a prompt like _"Open the mobile site on an emulated Pixel 9 running Android 14"_ to fall back to an emulator. See [Use cases & workflows](https://testingbot.com/support/ai/mcp/use-cases) for emulator vs real-device examples. ## Maestro / Espresso / XCUITest tools are missing or a run won't start ### Issue Your agent says it has no tool for running Maestro flows or an Espresso suite, or `uploadMaestroApp` and `runAppAutomateTest` reject the files or options you gave them. ### Cause & fix - **The tools don't exist at all** : App Automate support landed in `@testingbot/mcp-server` 1.1.0. If your client pinned an older version, update it. Running the server through `npx -y @testingbot/mcp-server@latest` always pulls the current release; a `.mcpb` bundle installed in Claude Desktop has to be re-downloaded from the [releases page](https://github.com/testingbot/mcp-server/releases). Restart the client afterwards so it re-reads the tool list. - **The upload is rejected** : each tool accepts specific file types. Maestro takes `.apk`, `.aab`, `.ipa` or a zipped `.app`. For Espresso, the app is an `.apk`/`.aab` and the test suite is the instrumented test `.apk`. For XCUITest, the app is an `.ipa` and the test suite is a `.zip` of the runner bundle. Give the agent absolute paths. - **An iOS run refuses to start** : `.ipa` builds need physical hardware. Set `realDevice: true` on `runMaestroTest` and `runAppAutomateTest` for iOS. - **Flows upload but nothing runs** : `uploadMaestroFlows` replaces the flows for a project, so a later upload that omitted some files leaves only what you sent last. Upload the whole directory, and check `includeTags`/`excludeTags`, a tag filter that matches nothing produces a run with no flows. - **Flows fail on the device but pass locally** : fetch `getMaestroFlowDetails` for the failing flow. It returns the error, the step-level report, and the video, screenshots and device logs, which is usually enough for the agent to spot a timing or selector problem. ### Runs that stay in WAITING `WAITING` means the run is queued for a device. A specific `deviceName` on busy hardware can queue for a while, so widen it (wildcards and regex are accepted, for example `Pixel 8*` or `*`), drop `realDevice` to use an emulator or simulator, or check that your plan has a free parallel slot. `cancelMaestroRun` stops a run you no longer want. ## The screenshot doesn't show in my chat ### Issue You asked for a screenshot with `tb_screenshot`, the tool reports success, but no image appears in your AI client's chat window. ### Cause `tb_screenshot` returns the screenshot _two_ ways: an inline image and a saved PNG file with a clickable `file://` link. Many MCP clients do not render inline tool images in the conversation, so the picture is captured correctly, your client just isn't displaying it. ### Fix - **Open the saved file** : the tool always returns a `file://` link to the PNG it wrote. Click or open that path to view the screenshot. - **Control where it's written** : pass a `savePath` (a `.png` file path or a directory) so the image lands somewhere you can find easily. Without it, the server saves to your OS temp directory. Example prompt: > "Take a screenshot of the current page and save it to ~/Desktop/checkout.png" ## Browser login didn't open, or I'm on SSH / a remote host ### Issue You ran `tb_login` but no browser window appeared, or you're working over SSH, in a container, or in a web-based client where opening a local browser isn't possible. ### Cause The default `tb_login` mode is `auto`, which prefers the **loopback** flow (RFC 8252): it opens a browser on your machine and captures credentials on a local `127.0.0.1` callback. On a headless or remote host there is no local browser to open, so that flow can't complete. ### Fix Use the **device-code** flow (RFC 8628) by running `tb_login` with `mode` set to `device`. The tool prints a short code; open [https://testingbot.com/device](https://testingbot.com/device) on any device, enter the code, and click Authorize. The flow is **resumable** , if it times out, just run `tb_login` again to finish. | mode | Best for | How it works | | --- | --- | --- | | `auto` (default) | Most setups | Loopback on a desktop, falls back to device on headless hosts. | | `loopback` | Local desktop | Opens your browser; credentials captured on a local 127.0.0.1 callback. Nothing pasted. | | `device` | SSH / remote / headless / web clients | Prints a code; you enter it at https://testingbot.com/device. Resumable. | You can also set a default with the `TESTINGBOT_AUTH_MODE` environment variable. More detail on [Authentication](https://testingbot.com/support/ai/mcp/authentication). ## Node version errors ### Issue The server fails to start, or `npx` throws syntax/engine errors when launching `@testingbot/mcp-server`. ### Cause The TestingBot MCP server requires a modern Node.js runtime. Older versions lack APIs the server (and its bundled `@testingbot/automation-mcp`) depend on. ### Fix Install **Node.js 18 or newer** (20+ recommended). Check your version, then confirm the install command your client uses: ```bash node --version # should print v18.x or higher (v20+ recommended) npx -y @testingbot/mcp-server@latest ``` If your MCP client uses a different Node than your shell (common with version managers like nvm), point the client's `command` at the correct Node binary, or make sure the right Node is first on your `PATH`. ## Sessions seem stuck or keep using minutes ### Issue An automation session appears to be running long after you're done with it, and you're worried it's still consuming TestingBot minutes. ### Cause Each `tb_openBrowser` call starts a real session that stays open until it's closed. If the agent moved on without calling `tb_closeBrowser`, the session can linger. ### Fix - **List active sessions** : ask the agent to run `tb_listSessions` to see everything currently open. - **Close them** : run `tb_closeBrowser` with the `sessionId` to end a session and free minutes immediately. - **Stop an account-level test** : for a non-automation session you can use `stopTest` with its `sessionId`. Idle sessions auto-close after a period of inactivity, so a forgotten session won't run indefinitely. but closing explicitly is faster and avoids wasted minutes. A good habit is to end a workflow with a prompt like _"close all my open browser sessions"_. ## Where are my logs ### Issue A test failed and you want to see what happened, selenium, browser, Appium or VM logs for the session. ### Cause Session logs live on TestingBot, not on your machine. The MCP server exposes them through the `getFailureLogs` tool rather than writing them locally. ### Fix Ask the agent to fetch logs for the session with `getFailureLogs`. You can filter by log type and limit the volume: | Param | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `sessionId` | string | Yes | – | The session to fetch logs for. | | `logTypes` | array | No | all | Any of `selenium`, `browser`, `chrome`, `vm`, `appium`. | | `failuresOnly` | boolean | No | false | Return only failure-related entries. | | `maxBytesPerLog` | number | No | 50000 | Cap per log (1000–1000000 bytes). | | `timeoutMs` | number | No | 15000 | Fetch timeout (1000–60000 ms). | A typical debugging flow is `getTests` → `getTestDetails` → `getFailureLogs`. See the worked example under [Use cases & workflows](https://testingbot.com/support/ai/mcp/use-cases), or the full parameter list in the [Tool reference](https://testingbot.com/support/ai/mcp/tools). ## Noisy output or WDIO\_LOG\_LEVEL ### Issue You're seeing unexpectedly verbose WebDriver output, or conversely you want more detail when diagnosing an automation problem. ### Cause The bundled automation layer uses a WebDriver client whose verbosity is controlled by `WDIO_LOG_LEVEL`. The server sets this to `silent` by default so your client's chat stays clean, so this rarely needs changing. ### Fix To get more detail while debugging, set `WDIO_LOG_LEVEL` in your client's `env` block (for example `error`, `warn`, `info` or `debug`). Leave it at `silent` for normal use. ```json { "mcpServers": { "testingbot": { "command": "npx", "args": ["-y", "@testingbot/mcp-server@latest"], "env": { "TESTINGBOT_KEY": "", "TESTINGBOT_SECRET": "", "WDIO_LOG_LEVEL": "info" } } } } ``` Still stuck? Double-check your install and config on the [Setup](https://testingbot.com/support/ai/mcp/setup) page, confirm you're authenticated via [Authentication](https://testingbot.com/support/ai/mcp/authentication), or review the [FAQ](https://testingbot.com/support/ai/mcp/faq). For an overview of the whole integration, see [TestingBot MCP](https://testingbot.com/support/ai/mcp). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/ai/mcp/faq # Frequently asked questions Answers to the most common questions about the [TestingBot MCP server](https://testingbot.com/support/ai/mcp): how it works with different AI assistants, how your data is handled, how authentication and billing work, and the difference between testing on emulators and real devices. If you are just getting started, see the [Setup & installation](https://testingbot.com/support/ai/mcp/setup) guide, the [Authentication](https://testingbot.com/support/ai/mcp/authentication) guide and the full [Tool reference](https://testingbot.com/support/ai/mcp/tools). Looking for in-browser, page-side AI tooling instead of the local MCP server? See [TestingBot WebMCP](https://testingbot.com/support/ai/webmcp), a different product. This page covers the local [TestingBot MCP server](https://testingbot.com/support/ai/mcp). ## General Questions about which AI clients the server works with and what you need installed locally. ### Is this MCP server compatible with various AI models or specific to a particular provider? MCP servers are LLM-agnostic, which means they work with any LLM model that supports the Model Context Protocol. ### Which MCP clients are supported? The TestingBot MCP server works with Claude Desktop, Claude Code, Cursor, VS Code (Copilot/MCP), Windsurf, and any MCP-compatible client that can launch a stdio server. One-click installers are available for VS Code (https://testingbot.com/mcp/install?client=vscode) and Cursor (https://testingbot.com/mcp/install?client=cursor), and Claude can install the bundled `.mcpb` from the GitHub releases page. For any other client, use the standard JSON config shown on the [Setup page](https://testingbot.com/support/ai/mcp/setup). ### Do I need to install Appium, browsers or device emulators locally? No. All browsers, emulators, simulators and real devices run on TestingBot's cloud. The only local requirement is Node.js 18+ (20+ recommended), which runs the MCP server itself via `npx -y @testingbot/mcp-server@latest`. Browser and mobile automation are handled by the bundled `@testingbot/automation-mcp` package, which drives remote sessions over WebDriver, you do not install Appium, Selenium, Chrome or any device tooling on your machine. ## Privacy & security How your prompts, data and credentials flow between your AI client, the MCP server and TestingBot. ### Is the information that I feed through MCP used to train? No, the MCP server does not send any information to an LLM. It's the LLM that is requesting information from the MCP server. The MCP server will perform API calls to the TestingBot API. ### Where are my credentials stored, and is it secure? When you log in with the `tb_login` tool, your TestingBot key and secret are written to `~/.testingbot/credentials`, an INI file with profile sections such as `[default]`. The directory is created with `0700` permissions and the file with `0600`, so only your user account can read it. Credentials never leave your machine: they are used only by the MCP server to call the TestingBot API directly, and are never passed to the LLM provider. See [Authentication](https://testingbot.com/support/ai/mcp/authentication) for full details. ### What is the difference between TestingBot MCP and WebMCP? TestingBot MCP is a local server you connect to your AI assistant (Claude, Cursor, VS Code, etc.) so it can manage your TestingBot account and drive real cloud browser and mobile sessions through natural language. [TestingBot WebMCP](https://testingbot.com/support/ai/webmcp) is a separate, in-browser approach. If you want your coding assistant or AI agent to interact with TestingBot's infrastructure, this MCP server is what you want. ## Authentication How the server signs in to TestingBot and how to provide credentials in different environments. ### How does authentication to TestingBot happen? There are three options. The easiest is to run the `tb_login` tool, which opens a browser login (loopback) on a desktop or prints a short device code for remote/headless setups, no key is ever pasted. Alternatively, set `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` in your client's `env` block (get them at https://testingbot.com/members/user/edit). The server resolves credentials in this order: environment variables (both must be set), then the credentials file, then degraded mode. See [Authentication](https://testingbot.com/support/ai/mcp/authentication). ### Why do the tools reply "No TestingBot credentials configured. Run the tb\_login tool"? That message means the server is running in degraded mode: it found no usable credentials in the environment or the credentials file, so every tool except `tb_login` is disabled. Run the `tb_login` tool to authenticate, or set `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` in your client config. On SSH or headless hosts, use the device-code flow, see [Troubleshooting](https://testingbot.com/support/ai/mcp/troubleshooting). ### The browser login did not open, I am on SSH or a remote host. What do I do? Use the device-code flow by running `tb_login` with `mode` set to `device`. It prints a short code; open https://testingbot.com/device in any browser, enter the code and click Authorize. The flow is resumable, if it times out, just run `tb_login` again to finish. The default `auto` mode already falls back to device login on headless machines. ## Devices & sessions How emulators, simulators and real devices differ, and how to watch or take over a live run. ### What is the difference between an emulator/simulator and a real device? Emulators (Android) and simulators (iOS) are virtual devices, they are cheaper and plentiful, so they are great for functional checks and CI. They are the default: `tb_openBrowser` with a `deviceName` and `platform` of Android or iOS uses `realDevice:false`. Set `realDevice:true` when you need physical hardware, real GPU, sensors and the real Chrome or Safari engine. Browse all available hardware at https://testingbot.com/support/app-automate/devices. You can also list devices through the agent with the `getDevices` tool. ### A requested real device is unavailable or busy, what happens? When you request a real device with `realDevice:true` and that device or OS version is busy or not offered, `tb_openBrowser` runs a pre-flight check and returns guidance _without_ starting (and billing) a session. The agent can then wait and retry, pick an available OS version it suggests, or fall back to an emulator/simulator. See [Use cases](https://testingbot.com/support/ai/mcp/use-cases) for the full workflow. ### Can I watch or take over a session the agent is running? Yes. Every automation session returns a live-view URL in the form `https://testingbot.com/tests//live?auth=`. Open it in your browser to watch the session in real time and interact with it directly when you need to step in. ## Usage, CI & teams Questions about billing, continuous integration and sharing the configuration with your team. ### Does using the MCP server consume my TestingBot minutes? Account and management actions, listing tests, fetching results, reading logs, managing builds or storage, do not consume testing minutes. Live and automation sessions do: opening a browser or device session (for example with `tb_openBrowser` or a live session tool) uses minutes while it runs. Close sessions with `tb_closeBrowser` when finished; idle sessions also auto-close. Use `tb_listSessions` to see what is currently open. ### Can I use the MCP server in CI or other automated environments? Yes. For CI and headless environments, provide credentials through environment variables instead of the interactive login. Set `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` in the `env` block of your MCP config: ```json { "mcpServers": { "testingbot": { "command": "npx", "args": ["-y", "@testingbot/mcp-server@latest"], "env": { "TESTINGBOT_KEY": "", "TESTINGBOT_SECRET": "" } } } } ``` Get your key and secret at https://testingbot.com/members/user/edit. If you cannot use env vars, the device-code login flow also works on remote hosts. ### Can team members share a single configuration? You can share the MCP config block itself (command and args), but each member should supply their own credentials. For shared or CI configs, an API key/secret in the `env` block is the most portable option. For local desktop use, each member runs `tb_login` once and credentials are written to their own `~/.testingbot/credentials` file. You can keep multiple sets of credentials side by side using profile sections and the `TESTINGBOT_PROFILE` environment variable. Team-level details (members, usage) are available through the `getTeam` and `getUsersInTeam` tools, see the [Tool reference](https://testingbot.com/support/ai/mcp/tools). Still stuck? The [Troubleshooting](https://testingbot.com/support/ai/mcp/troubleshooting) guide covers common errors such as degraded mode, unavailable devices, screenshots that do not render in chat, and Node version issues. ## App Automate: Maestro, Espresso & XCUITest Running native app test suites on devices through the agent. ### Can the agent run my Maestro, Espresso or XCUITest suite? Yes, from version 1.1.0 of `@testingbot/mcp-server`. The agent uploads your app and tests, starts a run on a device of your choosing, polls it, and reports back with the JUnit report plus video, screenshots and device logs for anything that failed. See the [Tool reference](https://testingbot.com/support/ai/mcp/tools) for the tools and their parameters, and [Use cases](https://testingbot.com/support/ai/mcp/use-cases) for the full sequences. ### Does the agent wait while my suite runs? No. App Automate runs are asynchronous by design: `runMaestroTest` and `runAppAutomateTest` return a run id straight away, and the agent polls `getMaestroRunStatus` or `getAppAutomateRunStatus` for progress. A suite that takes twenty minutes on devices does not hold your conversation open for twenty minutes. ### Can the agent write the Maestro flows for me? It can. `uploadMaestroFlows` accepts inline YAML as well as a directory or a `.zip`, so an agent can author a flow and upload it without touching your filesystem. The `maestroCheatSheet` tool exists for exactly this: agents fetch the Maestro syntax reference before writing or repairing a flow. The usual loop is write, run, read `getMaestroFlowDetails` for the failure, fix, retry. Our [Maestro documentation](https://testingbot.com/support/app-automate/maestro) covers the flow format in full. ### What is the difference between these tools and the Appium tools? The App Automate tools run a _pre-existing test suite_ (Maestro flows, an Espresso instrumentation APK, an XCUITest bundle) as a batch job on our device cloud, and hand you a report. The `appium_*` tools drive a single live app session step by step, with the agent deciding each tap and assertion as it goes. Use App Automate for regression suites, Appium for exploratory work and one-off interactions. ## More resources - [TestingBot MCP overview](https://testingbot.com/support/ai/mcp), what it is and what it can do. - [Setup & installation](https://testingbot.com/support/ai/mcp/setup), install and configure your AI client. - [Authentication](https://testingbot.com/support/ai/mcp/authentication), browser login, device code and API keys. - [Tool reference](https://testingbot.com/support/ai/mcp/tools), every available tool and its parameters. - [Use cases & workflows](https://testingbot.com/support/ai/mcp/use-cases), example prompts and tool sequences. - [Troubleshooting](https://testingbot.com/support/ai/mcp/troubleshooting), fixes for common issues. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/ai/webmcp # Testing WebMCP Tools WebMCP is a proposed browser API that lets a website expose its own actions as structured, callable tools to in-browser AI agents. Instead of an agent guessing where to click, your site registers tools with `navigator.modelContext.registerTool()`, each with a name, a description and a JSON Schema for its input. If your website ships WebMCP tools, you need to verify they keep working: that every tool is registered, that its input schema is valid, and that calling it returns the result you expect. This page shows how to run those checks on real browsers in the TestingBot cloud with Selenium, Playwright and Puppeteer. WebMCP is different from the [TestingBot MCP Server](https://testingbot.com/support/ai/mcp). The TestingBot MCP Server is a tool you install locally to let AI assistants (Claude, Cursor, VS Code) drive your TestingBot account through the REST API. WebMCP is a browser feature your own website implements so AI agents can call your site's actions. This page is about testing the latter. ## Why test your WebMCP tools WebMCP tools are part of your public interface, just like a form or an API endpoint. They break in the same ways: - **Registration** : a deploy removes or renames a tool, so the agent no longer finds it. - **Schema** : an input schema becomes invalid or stops matching what the tool actually accepts. - **Behavior** : the tool registers fine but its `execute()` callback returns the wrong result, throws, or silently changes shape. - **Cross-browser** : a tool works in one Chrome version but not another, or only when the page reaches a certain state. Running these checks on real browsers in CI catches regressions before your users' AI agents do. ## Prerequisites - A TestingBot account with your [key and secret](https://testingbot.com/members/user/security). - A website that registers WebMCP tools through `navigator.modelContext`. - Google Chrome 149 or higher (WebMCP is currently a Chromium-only origin trial). - The WebMCP testing surface enabled, so your test can list and invoke tools without a live AI agent. This exposes `navigator.modelContextTesting` with `getTools()` and `executeTool()`. WebMCP is experimental and runs behind Chrome feature flags during the origin trial. Launch Chrome 149 or higher with `--enable-features=WebMCPTesting,DevToolsWebMCPSupport`, which exposes `navigator.modelContextTesting` so your tests can list and invoke tools without a live agent. See the [Chrome WebMCP documentation](https://developer.chrome.com/docs/ai/webmcp) for details. If you would like WebMCP enabled on the TestingBot grid for your account, [contact support](https://testingbot.com/support). ## Testing WebMCP tools The pattern is the same in every framework: launch Chrome 149 or higher with the WebMCP testing flag, navigate to your page, then call `navigator.modelContextTesting.getTools()` to read the registered tools and `navigator.modelContextTesting.executeTool(name, input)` to invoke one and inspect its result. [Selenium (Python)](https://testingbot.com#) [Selenium (Node.js)](https://testingbot.com#) [Playwright](https://testingbot.com#) [Puppeteer](https://testingbot.com#) ```python from selenium import webdriver options = webdriver.ChromeOptions() options.browser_version = "latest" # Chrome 149 or higher # Enable the WebMCP testing surface (navigator.modelContextTesting) and the DevTools WebMCP domain options.add_argument("--enable-features=WebMCPTesting,DevToolsWebMCPSupport") options.set_capability("platformName", "WINDOWS") options.set_capability("tb:options", { "key": "YOUR_KEY", "secret": "YOUR_SECRET", "name": "WebMCP tool test", }) driver = webdriver.Remote( command_executor="https://hub.testingbot.com/wd/hub", options=options, ) try: driver.get("https://your-site.com") if not driver.execute_script("return 'modelContextTesting' in navigator"): raise RuntimeError("WebMCP testing surface is not available in this browser") # 1. List the tools your site registered with navigator.modelContext tools = driver.execute_async_script(""" const done = arguments[arguments.length - 1]; navigator.modelContextTesting.getTools().then(done); """) names = [t["name"] for t in tools] assert "searchProducts" in names, names # 2. Invoke a tool and check the structured result result = driver.execute_async_script(""" const [name, input, done] = arguments; navigator.modelContextTesting.executeTool(name, input).then(done); """, "searchProducts", {"query": "shoes"}) assert result["total"] > 0 finally: driver.quit() ``` ```javascript const { Builder } = require('selenium-webdriver') const chrome = require('selenium-webdriver/chrome') const options = new chrome.Options() options.setBrowserVersion('latest') // Chrome 149 or higher // Enable the WebMCP testing surface and the DevTools WebMCP domain (Chrome 149+) options.addArguments('--enable-features=WebMCPTesting,DevToolsWebMCPSupport') options.set('tb:options', { key: 'YOUR_KEY', secret: 'YOUR_SECRET', name: 'WebMCP tool test' }) ;(async () => { const driver = await new Builder() .usingServer('https://hub.testingbot.com/wd/hub') .forBrowser('chrome') .setChromeOptions(options) .build() try { await driver.get('https://your-site.com') // 1. List the registered tools const tools = await driver.executeAsyncScript(` const done = arguments[arguments.length - 1]; navigator.modelContextTesting.getTools().then(done); `) console.log(tools.map(t => t.name)) // 2. Invoke a tool and check its result const result = await driver.executeAsyncScript(` const [name, input, done] = arguments; navigator.modelContextTesting.executeTool(name, input).then(done); `, 'searchProducts', { query: 'shoes' }) console.log(result) } finally { await driver.quit() } })() ``` ```javascript const playwright = require('playwright') const capabilities = { 'tb:options': { key: 'YOUR_KEY', secret: 'YOUR_SECRET', name: 'WebMCP tool test' }, browserName: 'chrome', browserVersion: 'latest', // Chrome 149 or higher // Enable the WebMCP testing surface and the DevTools WebMCP domain (Chrome 149+) 'goog:chromeOptions': { args: ['--enable-features=WebMCPTesting,DevToolsWebMCPSupport'] } } ;(async () => { const browser = await playwright.chromium.connect( `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` ) const page = await browser.newPage() await page.goto('https://your-site.com') // 1. List the registered tools const tools = await page.evaluate(() => navigator.modelContextTesting.getTools()) console.log(tools.map(t => t.name)) // 2. Invoke a tool and check its result const result = await page.evaluate( ([name, input]) => navigator.modelContextTesting.executeTool(name, input), ['searchProducts', { query: 'shoes' }] ) console.log(result) await browser.close() })() ``` ```javascript // Puppeteer 25.1.0 or higher ships a native page.webmcp API const puppeteer = require('puppeteer-core') const capabilities = { 'tb:options': { key: 'YOUR_KEY', secret: 'YOUR_SECRET', name: 'WebMCP tool test' }, browserName: 'chrome', browserVersion: 'latest', // Chrome 149 or higher // Enable the WebMCP testing surface and the DevTools WebMCP domain 'goog:chromeOptions': { args: ['--enable-features=WebMCPTesting,DevToolsWebMCPSupport'] } } ;(async () => { const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) const page = await browser.newPage() await page.goto('https://your-site.com') // 1. List the tools your site registered with navigator.modelContext const tools = page.webmcp.tools() console.log(tools.map(t => t.name)) // 2. Invoke a tool and check its structured result const tool = tools.find(t => t.name === 'searchProducts') if (tool) { const result = await tool.execute({ query: 'shoes' }) if (result.status === 'Completed') { console.log(result.output) } } await browser.close() })() ``` Older Puppeteer versions, and Selenium and Playwright, do not have a native WebMCP API. There you can reach the same tools by evaluating `navigator.modelContextTesting.getTools()` and `executeTool()` in the page, as shown in the other tabs. After the run, mark the test as passed or failed in your TestingBot dashboard the same way you do for any other test, either from your test script or through the [REST API](https://testingbot.com/support/api). ## What to assert A useful WebMCP test suite checks more than "the page loaded". Good assertions include: - **Tool presence** : every tool you expect is listed by `getTools()`, with the right name and description. - **Schema correctness** : each tool's `inputSchema` declares the parameters you intend, including required fields and enums. - **Happy path** : `executeTool()` with valid input returns the expected structured result. - **Error handling** : invalid input is rejected cleanly rather than throwing an unhandled error. - **State changes** : write tools (for example "add to cart") actually change the page state, and read-only tools do not. ## Browser support and caveats WebMCP is an early, evolving standard. It is a W3C Community Group draft, not yet on the standards track, and is currently implemented only in Chromium-based browsers behind an origin trial (Chrome 149 and higher). The API has changed several times, so treat any integration as experimental and isolate your WebMCP test helpers so you can update them when the API moves. - WebMCP is Chromium-only today. Firefox and Safari have not committed to it. - The exact flag and API names can change between Chrome releases during the origin trial. - Your site must enable WebMCP (through its origin-trial token or flag) for `navigator.modelContext` to be available. ## FAQ ### What is the difference between WebMCP and the TestingBot MCP Server? They are unrelated. WebMCP is a browser API your website implements so in-browser AI agents can call your site's actions. The [TestingBot MCP Server](https://testingbot.com/support/ai/mcp) is a separate tool you install locally so AI assistants can manage your TestingBot account through the REST API. This page covers testing the WebMCP tools your own site exposes. ### Which browsers support WebMCP? WebMCP is currently available only in Chromium-based browsers (Chrome 149 and higher) as an origin trial. Firefox and Safari have not committed to it yet. You can run your tests on Chrome in the TestingBot cloud today. ### Do I need an AI agent to test my WebMCP tools? No. The WebMCP testing surface (`navigator.modelContextTesting`) lets your test list and invoke tools directly, so you can assert on the results in regular Selenium, Playwright or Puppeteer tests without involving a live AI model. ### Is WebMCP production ready? Not yet. WebMCP is an early standard in an origin trial and the API is still changing. It is a good time to experiment and to start testing your tools, but keep your WebMCP test helpers isolated so they are easy to update as the standard evolves. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-live # Live Browser and Mobile Testing Live testing lets you drive a remote browser, simulator or real mobile device in the TestingBot cloud with your own mouse and keyboard, exactly as if it were running locally. Below are answers to the questions we get most often about Live testing. For automated test execution see [Web Automate](https://testingbot.com/support/web-automate) instead. ## Is usage limited in terms of minutes or users? With an upgraded plan you can use as many Live testing minutes as you like, with as many co-workers as your seat count allows. There are no per-test or per-day limits. Trial accounts can use up to 60 Live testing minutes in total. See [our pricing plans](https://testingbot.com/pricing) for current seat and concurrency limits. ## What do I get with a free trial? Trial accounts can use up to 60 free minutes of Live testing. Once you open a browser in our cloud, you get 5 minutes of unlimited interactive access to that session. With an upgraded plan, Live testing minutes are unlimited within your plan. ## Is Live testing on TestingBot secure? None of your data is stored on the test VMs. Every Live session runs on a freshly-provisioned virtual machine that is destroyed when the session ends. VMs use up-to-date security patches and run inside a hardened EU datacenter behind firewalls. See [our security and compliance overview](https://trust.testingbot.com/) for GDPR and CSA STAR details. ## How can I test my staging site or a website on my own network? You can use TestingBot to test a staging or internal site that is not publicly reachable. Start the [TestingBot Tunnel](https://testingbot.com/support/tunnel) on your local machine or in CI, pick a browser in the dashboard, and the remote browser will reach your private URLs through the encrypted tunnel as if it were on your network. ## Do I need to download any software? No. You do not need to install any software, plugin or browser extension. In the member area, pick the browser, version and platform you want to test on. Once we boot a virtual machine for you, you control the machine from inside your own browser. For example, you can drive a remote Safari on macOS from a local Chrome on Windows. ## Which browsers and platforms do you provide for Live testing? The current and recent versions of Chrome, Firefox, Safari and Microsoft Edge on Windows, macOS and Linux, plus mobile Safari on real iPhones and iPads and mobile Chrome on real Android devices and emulators. Legacy browsers including older Internet Explorer versions are still available for compliance and regression work. See the full [list of supported browsers](https://testingbot.com/support/web-automate/browsers) for the current versions on the grid. ## What developer tools are pre-installed on the browsers? Every Live browser has its built-in developer tools available: Chrome DevTools, Firefox DevTools, Safari Web Inspector, and Edge DevTools. Open them with F12 or the usual keyboard shortcut. The remote desktop also includes standard system utilities for network inspection. No extension installation is required. ## Is audio enabled during testing? Audio is enabled on all desktop Live sessions and on iOS simulators and Android emulators. On real Android devices, audio is available on devices running Android 12 or higher. On real iOS devices, audio is available on a select group of devices running iOS 14 or higher. ## Can I embed Live testing in my own website or product? Yes, you can embed TestingBot's Live testing interface in your own app or website using an iframe. This is useful for QA dashboards, internal tools and bug-reporting flows that need an interactive remote browser. 1. Start a Live session through an API call. 2. Fetch the `sessionId` of the session and generate an `authentication hash`. 3. Pass the `sessionId` and `authentication hash` to a liveview URL and embed it with an iframe. ```javascript const TestingBot = require('testingbot-api'); const tb = new TestingBot({ api_key: process.env.TESTINGBOT_KEY, api_secret: process.env.TESTINGBOT_SECRET }); const options = { capabilities: { browserName: 'chrome', browserVersion: 'latest', platform: 'WIN11', 'tb:manual': true, } }; const session = await tb.createSession(options); const sessionId = session.session_id; const authHash = tb.getAuthenticationHashForSharing(sessionId); const liveViewUrl = `https://testingbot.com/tests/${sessionId}/live?auth=${authHash}`; ``` You can then embed the live view URL in an iframe as follows: ```html ``` The result is a fully functional Live testing interface embedded in your own application. For enterprise embed deals and dedicated infrastructure, [contact our team](https://testingbot.com/contact/new). ## Next steps - [Live ChromeOS testing](https://testingbot.com/support/web-live/chromeos). Test on a real Chromebook with mouse and keyboard. - [Browser extension testing](https://testingbot.com/support/web-live/extension). Install and test Chrome, Firefox and Edge extensions on the live browser. - [Live app testing](https://testingbot.com/support/app-live). Interactive testing on real iPhones, iPads and Android phones. - [Web Automate](https://testingbot.com/support/web-automate). Run Selenium, Playwright, Puppeteer and Cypress tests on the same grid. - [TestingBot Tunnel](https://testingbot.com/support/tunnel). Test internal and staging sites. - [Pricing](https://testingbot.com/pricing). Live and Automate plans, seat counts and concurrency limits. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-live/extension # Browser Extension - [What is the TestingBot Browser Extension?](https://testingbot.com#usage) - [Installation](https://testingbot.com#installation) - [Using the Browser Extension](https://testingbot.com#usage) ## What is the TestingBot Browser Extension? The TestingBot Browser Extension can be installed on your [Chrome](https://chrome.google.com/webstore/detail/testingbot/nhjmiaiamlfblfagaodhkeabfbnmdmff) or [Firefox](https://addons.mozilla.org/addon/testingbot/) browser, for free. Once installed, you can visit any website with your own Chrome or Firefox browser, and click the TestingBot extension icon. A window will pop up, allowing you to choose on which remote browser you want to test the current page on. Upon selecting a browser, the current page will be available for testing on a remote TestingBot browser. For example, you are using Chrome but want to make sure that the website you are currently seeing looks and functions correctly on a macOS Safari browser. With the extension, you can simply choose the remote browser configuration. You will be redirected to a live view of the remote browser, where you can use your mouse and keyboard to interact and test the webpage. ## Installation The TestingBot Browser Extension is currently available for [Chrome](https://chrome.google.com/webstore/detail/testingbot/nhjmiaiamlfblfagaodhkeabfbnmdmff) and [Firefox](https://addons.mozilla.org/addon/testingbot/). Simply click the **Add Extension** button to add the extension to your browser. A TestingBot icon will now appear in your browser's navigation bar. You can pin the extension to make sure it remains visible at all times. ## Using the Browser Extension After installation, you can simply navigate to any website you want to test. Click the TestingBot extension icon to open a popup with browser choices. ![TestingBot Browser Extension](https://testingbot.com/assets/support/manual/extension-201001ac8ef0e3e987d2163148ebaff5c6062c9f3dbc2c6c0077cd16b1e4dd15.webp) Make sure you are logged in to TestingBot while using the extension. --- URL: https://testingbot.com/support/web-live/chromeos # ChromeOS Manual Testing TestingBot gives you the ability to test your website on a remote ChromeOS device emulator, such as a Chromebook. Simply enter the URL of the website you want to test and choose between 3 different ChromeOS configurations: - ChromeOS Small: 1366x768 - 160dpi - ChromeOS Medium: 3840x2160 - 320dpi - ChromeOS Large: 1920x1080 - 160dpi ## Starting a live ChromeOS session Simply [start a new manual session](https://testingbot.com/members/manual/new) by clicking the Android category and choosing one of the three ChromeOS configurations. A remote session will be started and you will be able to interact with the ChromeOS device emulator with your own mouse and keyboard. An audio stream will be started, so you can hear the sounds from the ChromeOS device emulator. ![Test on ChromeOS](https://testingbot.com/assets/support/manual/chromeos-7d003af4076458e52d6a4381b4bf2fee154591253bfe1070e9af5583c4256cac.jpg) ## Enabling/disabling Dark Mode You can easily enable or disable Dark Mode in the ChromeOS emulator by clicking the "Device Settings" button in taskbar, then toggling the "Enable Dark Mode" switch. ## Taking screenshots To take a live screenshot of the current ChromeOS session, simply click the "Photo" button in the taskbar. This will capture the current view and open the screenshot in a photo editor, which you can use to annotate or modify the screenshot, before downloading or sharing it. ## Change GPS You can use the "GPS" button in the taskbar to change the GPS location of the ChromeOS device emulator. This is useful for testing location-based features on your website. --- URL: https://testingbot.com/support/app-live # Live Mobile App Testing Take control over our real devices with your mouse and keyboard, straight from your own browser, without having to install any plugin or program. Simply upload your iOS or Android app to TestingBot, pick the device you want to test on, and TestingBot will give you a live session with the real device. ## [Upload App](https://testingbot.com/support/app-live/upload-app) Upload, manage, and delete your iOS and Android apps via the UI or API. ## [Select Device](https://testingbot.com/support/app-live/select-device) Choose from real devices and virtual devices. Understand time limits and device options. ## [Test Interface](https://testingbot.com/support/app-live/test-interface) Explore toolbar actions: screenshots, rotation, clipboard, GPS, locale, and more. ## [Developer Tools](https://testingbot.com/support/app-live/developer-tools) App Inspection, Appium Inspector, and real-time device logs for debugging. ## [Local Testing](https://testingbot.com/support/app-live/local-testing) Test apps that connect to your local network using TestingBot Tunnel. ## [Platform-Specific](https://testingbot.com/support/app-live/chromeos) ChromeOS app testing and Apple Pay testing on real devices. ## Developer Tools TestingBot provides powerful developer tools built into the live testing interface. Inspect elements, find locators, view real-time logs, and use the full Appium Inspector — all without leaving your session. [Explore Developer Tools →](https://testingbot.com/support/app-live/developer-tools) ## Audio Audio is available during manual mobile app testing, for certain devices. ### Virtual Devices Audio is available for all Android emulators and iOS Simulators. TestingBot will stream the audio generated by the virtual device to your web browser. ### Physical Devices Android devices with Android version 12 and higher will stream the audio to your browser, you will hear all the audio generated on the device. Older physical devices do not have the audio streaming capabilities. These will only stream a live video view of the device. ## Developer Resources - [API Reference](https://testingbot.com/support/api) - [Integrations](https://testingbot.com/integrations) - [Devices](https://testingbot.com/support/app-automate/devices) - [TestingBot Tunnel](https://testingbot.com/support/tunnel) - [Features](https://testingbot.com/features/manual-mobile-testing) - [Private Devices](https://testingbot.com/enterprise/private-device-cloud) ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-live/upload-app # Uploading a Mobile App There are several ways to get your app onto a TestingBot device for live testing. You can upload your own app file, install directly from the App Store or Play Store, use TestFlight for beta apps, or upload programmatically via our API. ## Upload via the UI To upload a mobile app through the TestingBot UI, simply go to the member dashboard and: - Click **Live App Testing** - Under **Select Source** click **Uploaded Apps** - Click **Upload App** to browse for the file ([see supported formats](https://testingbot.com#formats)) ## Other App Sources Besides uploading your own app file, you can install apps directly from these sources: ### [App Store](https://testingbot.com/support/app-live/app-store) Install iOS apps directly from the Apple App Store on real iOS devices. ### [Play Store](https://testingbot.com/support/app-live/play-store) Install Android apps directly from the Google Play Store on real Android devices. ### [TestFlight](https://testingbot.com/support/app-live/testflight) Install beta iOS apps via Apple TestFlight on supported real devices. ## Upload via the API You can programmatically upload apps using the [TestingBot REST API](https://testingbot.com/support/api/storage#upload). This is useful for CI/CD pipelines and automated workflows. ```bash curl -X POST "https://api.testingbot.com/v1/storage" \ -u "YOUR_KEY:YOUR_SECRET" \ -F "file=@/path/to/your/app.apk" ``` The API returns a unique `app_url` that you can use to start a live testing session or pass to your automated tests. See the full [API documentation](https://testingbot.com/support/api/storage#upload) for more details, including how to list and delete uploaded apps. ## Supported File Formats Currently we accept these file formats: - `.aab`, `.apk` for Android apps - `.ipa` for physical iOS devices, `.zip` for iOS simulators ## Deleting a Mobile App To delete a mobile app, you can use the TestingBot UI or use our [REST API](https://testingbot.com/support/api/storage#filedelete). ### Delete via the UI To delete a mobile app through the TestingBot UI, simply go to the member dashboard and: - Click **Live App Testing** - Under **Select Source** click **Uploaded Apps** - Click the trash icon in the bottom right corner of the list - Select one or more apps you wish to delete - Click **Delete Apps** ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-live/app-store # Install Apps from the App Store TestingBot allows you to install iOS apps directly from the Apple App Store onto real iOS devices during a live testing session. This lets you test production apps without needing the source code or an .ipa file. ## How to Install an App from the App Store To install an app from the App Store on a TestingBot device: - Go to the [Live App Testing](https://testingbot.com/members/apps/devices/new) page - Under **Select Source** , click **App Store** - Sign in with your Apple ID - Search for the app you want to install - Click **Install** and select a device to start your testing session ## Requirements - A valid **Apple ID** is required to sign in to the App Store - The app must be available in the App Store (not removed or region-restricted) - Only available on **real iOS devices** (not simulators) ## Limitations - Apps that require in-app purchases may have limited functionality during testing - Some apps may have DRM restrictions that prevent installation on shared devices - Your Apple ID credentials are only used during the session and are not stored by TestingBot ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-live/play-store # Install Apps from the Google Play Store TestingBot allows you to install Android apps directly from the Google Play Store onto real Android devices during a live testing session. This lets you test production apps without needing the source code or an .apk file. ## How to Install an App from the Play Store To install an app from the Google Play Store on a TestingBot device: - Go to the [Live App Testing](https://testingbot.com/members/apps/devices/new) page - Under **Select Source** , click **Play Store** - Sign in with your Google account - Search for the app you want to install - Click **Install** and select a device to start your testing session ## Requirements - A valid **Google account** is required to sign in to the Play Store - The app must be available on the Google Play Store - Only available on **real Android devices** (and some Android emulators) ## Limitations - Some apps may not be available on all device models due to compatibility restrictions set by the developer - Apps that require in-app purchases may have limited functionality during testing - Your Google account credentials are only used during the session and are not stored by TestingBot ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-live/testflight # Install Beta Apps via TestFlight TestingBot allows you to install beta iOS apps via Apple TestFlight on real devices. This is useful for testing pre-release versions of your app before they go live on the App Store. ## How to Install an App via TestFlight To install a beta app via TestFlight on a TestingBot device: - Go to the [Live App Testing](https://testingbot.com/members/apps/devices/new) page - Under **Select Source** , click **TestFlight** - Sign in with your Apple ID - You will see a list of beta apps you have been invited to test - Click **Install** on the app you want to test, then select a device ## Requirements - A valid **Apple ID** is required to sign in to TestFlight - You must have been **invited as a tester** by the app developer, or have a public TestFlight link - Only available on **real iOS devices** (not simulators) - TestFlight is pre-installed on TestingBot's real iOS devices ## Limitations - TestFlight beta apps have an expiration date set by the developer - Not all devices may be compatible with every beta build, depending on the developer's configuration - Your Apple ID credentials are only used during the session and are not stored by TestingBot ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-live/select-device # Selecting a Mobile Device Once you've selected a mobile app, you will need to select the device you want to run tests on. You can choose to select a Virtual Device or a Real Device by using the navigation tabs on the page. ## Real Devices To use a Real Device, click the [Real Device Testing](https://testingbot.com/members/apps/devices/new) tab. - Click an available device from the devices list - Make sure to select an iOS device if you've selected an iOS app (`.ipa`), or an Android device if you've selected an Android app (`.apk`) - Click the **Start** button ## Virtual Devices To use a Virtual Device, click the [Emulator & Simulator Testing](https://testingbot.com/members/apps/new) tab. - Select an app you've uploaded, or the sample Android/iOS TestingBot app. - Click the Android or iOS tab in the virtual device list - Select an iOS simulator if you've selected an iOS app (`.ipa`), or an Android emulator if you've selected an Android app (`.apk`) - Click the **Start** button ## Time Limits and Timeouts - Live tests will last a maximum of 5 minutes at a time for trial users. - Live tests for paid users will timeout after 15 minutes of inactivity ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-live/test-interface # Live Test Interface The Live Test Interface provides a comprehensive set of tools for interacting with your mobile app during live testing sessions. You can take screenshots, rotate the device, copy to clipboard, simulate GPS locations, change locale settings and more. ## Toolbar Actions | Icon | Name | Description | | --- | --- | --- | | | Switch Device | Change to a different device. This will start a new session, you will lose the session with the current device. | | | Zoom | Zoom in or out. This will make the device's screen appear larger or smaller. This only affects the view in your browser, it does not change anything on the device itself. | | | Rotate | Rotates the device. Either from PORTRAIT to LANDSCAPE or vice-versa. | | | Report Bug | Captures a bug: takes a screenshot from the device and opens an editor where you can annotate the image. Use **Report a bug** to create an issue in your bug tracker (Jira, GitHub or Bugsnag) or share the screenshot via e-mail or Slack, or use **Download** to save it locally. | | | Copy/Paste | Copy or paste from or to the device. If you've selected the copy/paste permission in your browser, TestingBot will automatically keep your local and remote clipboard in sync. | | | DevTools | This will open the developer tools for the device. Retrieve realtime logs from the device. Use the **Inspector** functionality to drill down in the view hierarchy to copy element locators (such as XPath). Learn more: [App Inspection](https://testingbot.com/support/app-live/app-inspection), [Appium Inspector](https://testingbot.com/support/app-live/appium-inspector), [Device Logs](https://testingbot.com/support/app-live/device-logs) | | | GPS Location | Set a (new) mock location for the GPS of the device. If your app uses GPS functionality, it will receive this mock location | | | Locale | Set a new locale for the entire device. Useful if your app is multilingual. Test your app in different languages. | | | Device Settings | Change various device settings. Switch between light and dark mode instantly. | | | Stop Session | Stops the current session. Logs and a video of the test will be available in the TestingBot member area. | ## Developer Tools The DevTools button provides access to powerful debugging features. Learn more about each tool: - [App Inspection](https://testingbot.com/support/app-live/app-inspection) — Find element locators by clicking on elements or searching the tree - [Appium Inspector](https://testingbot.com/support/app-live/appium-inspector) — Full Appium Inspector built into TestingBot - [Device Logs](https://testingbot.com/support/app-live/device-logs) — Real-time log streaming with filtering and download ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-live/developer-tools # Developer Tools TestingBot provides powerful developer tools built into the live testing interface. Access them by clicking the **DevTools** icon in the toolbar during any live session. ## [App Inspection](https://testingbot.com/support/app-live/app-inspection) Click on any element on the device screen to see its locators, or search the element tree by text or attribute. ## [Appium Inspector](https://testingbot.com/support/app-live/appium-inspector) The full Appium Inspector built directly into TestingBot. Inspect elements, view properties, and perform actions without any setup. ## [Device Logs](https://testingbot.com/support/app-live/device-logs) Stream real-time device logs, filter by severity level, search through entries, and download logs for offline analysis. ## Accessing Developer Tools During any live app testing session, click the **DevTools** icon in the toolbar on the left side of the screen. This will open a panel where you can switch between the available tools. ## Developer Resources - [API Reference](https://testingbot.com/support/api) - [Integrations](https://testingbot.com/support/integrations) - [Devices](https://testingbot.com/support/app-automate/devices) - [TestingBot Tunnel](https://testingbot.com/support/tunnel) - [Device Features](https://testingbot.com/real-device-testing) - [Private Devices](https://testingbot.com/enterprise/private-device-cloud) ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-live/app-inspection # App Inspection App Inspection allows you to find element locators directly during a live testing session. This is useful when building automated tests — you can identify the exact locators (XPath, Accessibility ID, etc.) for any element on the screen. ## Accessing App Inspection To open App Inspection during a live session: - Click the **DevTools** icon in the toolbar - Select the **Inspector** tab ## Click to Inspect Click on any element on the live device screen to immediately see its locators. The inspection panel will display all available locator strategies for the selected element. ## Search the Element Tree Use the search bar at the top of the inspection panel to find elements by text content or attribute value. This is helpful when you need to locate elements that are off-screen or deeply nested in the view hierarchy. ## Available Locators When you select an element, the following locator strategies are displayed: | Locator | Description | | --- | --- | | XPath | Full XPath expression to uniquely identify the element in the view hierarchy. | | Accessibility ID | The accessibility identifier set by the app developer. Preferred locator for stable tests. | | Class Name | The UI element class (e.g., `android.widget.Button`, `XCUIElementTypeButton`). | | ID | The resource ID of the element (Android) or the element name (iOS). | ## Copy Locators Click the copy icon next to any locator to copy it to your clipboard. You can then paste it directly into your automated test code. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-live/appium-inspector # Appium Inspector TestingBot includes the full Appium Inspector built directly into the live testing interface. This provides the same functionality as the standalone Appium Inspector desktop application, without requiring any local setup. ## Accessing Appium Inspector To launch the Appium Inspector during a live session: - Click the **DevTools** icon in the toolbar - Select the **Appium Inspector** tab ## Interface Overview The Appium Inspector interface consists of: - **Device Screenshot** — A snapshot of the current device screen with clickable elements - **Element Tree** — A hierarchical view of all UI elements on the screen - **Element Properties** — Detailed attributes and locators for the selected element - **Actions Panel** — Buttons to perform actions on selected elements ## Inspecting Elements Click on any element in the device screenshot or in the element tree to select it. The properties panel will show all available attributes and locators for the element. You can copy any locator value with a single click for use in your automated test scripts. ## Element Actions The Appium Inspector allows you to perform actions directly on selected elements: - **Tap** — Tap on the selected element - **Send Keys** — Type text into input fields - **Swipe** — Perform swipe gestures on the element These actions are executed on the real device in real-time, so you can see the result immediately on the live device screen. ## Advantages - **No setup required** — No need to install the standalone Appium Inspector or configure connection settings - **Works with any device** — Available for all real devices and virtual devices in the TestingBot device cloud - **Session-aware** — Automatically connected to your current live testing session - **Real-time** — The element tree updates as you interact with the device ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-live/device-logs # Device Logs TestingBot provides a real-time stream of the device logs during the manual live session. ## Log Levels You can switch between these device log levels: - `VERBOSE` (default) - `DEBUG` - `INFO` - `WARNING` - `ERROR` ## Search & Filter You can use the search input bar to search between the generated logs. The search filters the log entries in real-time as you type, making it easy to find specific messages or errors. ## Log Actions | Icon | Functionality | | --- | --- | | | The download icon allows you to download the entire device logs generated thus far, without any filtering. | | | The trash icon will clear the history of the logs. Only new log entries will be added to the realtime view. | ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-live/local-testing # Testing Local Websites/Apps With our [TestingBot Tunnel](https://testingbot.com/support/tunnel) you can easily and securely connect your local computer/network with TestingBot's Real Devices. ## Known Limitations - **Localhost testing does not work on iOS**. Due to an iOS Restriction, it is currently not possible to run tests against `http(s)://localhost` with TestingBot Tunnel. A possible workaround would be to change the hosts file on the machine running TestingBot Tunnel, and add a custom hostname (for example `mylocalwebsite` and the IP address `127.0.0.1`). Then configure your tests to use `http(s)://mylocalwebsite/...` instead of localhost. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-live/apple-pay # Testing Apple Pay Apple Pay, developed by Apple Inc., serves as a mobile payment and digital wallet service enabling transactions through various Apple devices, such as iPhones and iPads. Testing with Apple Pay comes with challenges. This document aims to clarify some methods that are available for testing with iOS and Apple Pay. There are important differences between Apple Pay on Real Devices and iOS Simulators. The iOS Simulator has the following limitations: - The simulator only targets the front-end integration of Apple Pay and does not test the back-end integration. - You can not add payment cards to the wallet on iOS simulators. So no Apple Pay sandbox testing cards, or any other real credit card. - It is not possible to test the Apple Pay in-web flow. - It is possible to test the Apple Pay in-app flow, but the Apple Pay in-app flow will not work the same way as with a real iOS device. It will not return a payment token and will not properly process a payment. ## Apple Pay on Real Private Devices TestingBot provides [private iOS devices](https://testingbot.com/enterprise/private-device-cloud), which are able to use Apple Pay. To make Apple Pay work on TestingBot's real private devices, please follow the steps below: 1. Follow Apple's documentation to [enable Apple Pay](https://developer.apple.com/documentation/passkit/apple_pay/setting_up_apple_pay) 2. Set up Apple Pay integration in your iOS app, using the Apple documentation. 3. Register your Merchant ID in your Apple developer account. 4. Set up an Apple sandbox tester account. See the [Apple Sandbox documentation](https://developer.apple.com/help/app-store-connect/test-in-app-purchases/create-sandbox-apple-ids/) for more information. 5. Build your ipa file. Apple Pay will not work with enterprise certificates, which means it will not work by default on TestingBot's private devices. You will need to add the `udid` (iPhone/iPad unique id) of each private device assigned to you, to your Apple provisioning profile. - Log in to [Apple's Developer Account](https://developer.apple.com/account/resources/identifiers/list) - Click [Add Device](https://developer.apple.com/account/resources/devices/add) - Type in a device name and the UDID that you received from TestingBot. - Save the device. - Now add the new device to your [Provisioning Profile](https://developer.apple.com/account/resources/profiles/list) ### Passcode One of the requirements to use Apple Pay, is that a passcode needs to be set on the device. Make sure that you set a passcode on your private iOS device. TestingBot cleans every private device after each test session. You will need to configure your Apple Pay Sandbox Testing account, including a passcode and sandbox cards, every time you want to test Apple Pay on a real iOS device. If you want to disable this automated cleaning process, please [reach out](https://testingbot.com/contact/new). ## Apple Pay on iOS Simulators iOS Simulators offer a feature where you can authorize Apple Pay by clicking a button in the Simulator's menu bar. - Navigate to a point in your app where a payment sheet is visible. - Click **Features** , then enable **Authorize Apple Pay** , or use the key combination (Command+SHIFT+A). ## Apple Sandbox Test Cards You can find a list of Test Cards and their details on [Apple Pay's Sandbox Testing documentation page](https://developer.apple.com/apple-pay/sandbox-testing/). These test cards include Discover, Master Card, American Express, VISA and others. There are cards available for the USA, Europe, China and even Test Cards for Point of Sale Systems. To use these, simply open the **Wallet** application on your iOS device and add the card number and expiration date. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-live/chromeos # ChromeOS Manual App Testing With TestingBot, you can upload your native Android app and test it on a remote ChromeOS device emulator, such as a Chromebook. After you upload your app, simply choose one of the three ChromeOS configurations to start testing: - ChromeOS Small: 1366x768 - 160dpi - ChromeOS Medium: 3840x2160 - 320dpi - ChromeOS Large: 1920x1080 - 160dpi ## Starting a live ChromeOS session To get started, make sure you uploaded a `.apk` file on the [manual app session](https://testingbot.com/members/apps/new) page. Then, click one of the three ChromeOS configurations to start a new manual session. A remote session will be started and you will be able to interact with the ChromeOS device emulator with your own mouse and keyboard. An audio stream will be started, so you can hear the sounds from the ChromeOS device emulator. ![Test an app on ChromeOS](https://testingbot.com/assets/support/manual/chromeos-app-1e807772235f4f2af5ee949006a49b10bca80b6ea7f4f40b6118aa5c21124d56.jpg) ## Enabling/disabling Dark Mode You can easily enable or disable Dark Mode in the ChromeOS emulator by clicking the "Device Settings" button in taskbar, then toggling the "Enable Dark Mode" switch. ## Taking screenshots To take a live screenshot of the current ChromeOS session, simply click the "Photo" button in the taskbar. This will capture the current view and open the screenshot in a photo editor, which you can use to annotate or modify the screenshot, before downloading or sharing it. ## Change GPS You can use the "GPS" button in the taskbar to change the GPS location of the ChromeOS device emulator. This is useful for testing location-based features on your website. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/api # TestingBot API Documentation Access and modify all your TestingBot data through our comprehensive REST API. Build powerful integrations and automate your testing workflows. - **Endpoint:** api.testingbot.com - **Version:** v1 - **Format:** JSON - **Auth:** HTTP Basic [Download OpenAPI 2.0 spec (JSON)](https://testingbot.com/api/v1/openapi.json) ## API Reference The reference is split by resource. Every endpoint below is authenticated with the API key and secret described in [Authentication](https://testingbot.com#authentication). - [Browsers & Devices 5 endpoints Discover which browsers, operating systems and real devices you can run on, and look up the IP ranges to allowlist.](https://testingbot.com/support/api/devices) - [User & Team Management 16 endpoints Your own account and credentials, plus everything needed to provision team members and service accounts from a script.](https://testingbot.com/support/api/user) - [Tests & Builds 17 endpoints Fetch test results, report pass/fail state back from your CI, and work with the builds that group them.](https://testingbot.com/support/api/tests) - [Insights 5 endpoints The data behind the Test Analytics dashboard, exposed so you can build your own reporting.](https://testingbot.com/support/api/insights) - [Screenshots 3 endpoints Kick off a screenshot run across browsers and fetch the images when it finishes.](https://testingbot.com/support/api/screenshots) - [Tunnel 7 endpoints Manage the secure tunnels that let the grid reach your staging and internal environments.](https://testingbot.com/support/api/tunnel) - [Codeless Automation & TestLab 29 endpoints Drive the codeless test builder and the TestLab suites that group those tests, end to end.](https://testingbot.com/support/api/codeless) - [TestingBot Storage 5 endpoints Upload app binaries and test bundles once, then reference them from your capabilities with a tb:// URL.](https://testingbot.com/support/api/storage) - [Webhooks 7 endpoints Manage the callbacks TestingBot posts when a test finishes, and verify them without waiting for a run.](https://testingbot.com/support/api/webhooks) - [App Automate 32 endpoints Everything for native mobile test runs: Maestro flows, Espresso and XCUITest suites, and their reports.](https://testingbot.com/support/api/app-automate) ## API Clients There are several open source client libraries available to easily interact with the TestingBot API: - [NodeJS](https://github.com/testingbot/testingbot-api) - [Python](https://github.com/testingbot/testingbotclient) - [Java](https://github.com/testingbot/testingbot-java) - [Ruby](https://github.com/testingbot/testingbot_ruby) - [PHP](https://github.com/testingbot/testingbot-php) - [.NET](https://github.com/testingbot/testingbot-dotnet) ## Authentication Every TestingBot API request is authenticated with your **API key** and **API secret** using [HTTP Basic Auth](http://en.wikipedia.org/wiki/Basic_access_authentication) over HTTPS. The only unauthenticated endpoint is [GET /v1/browsers](https://testingbot.com#browsers). ### 1 · Find your credentials Sign in to TestingBot and open [Account → Account Info](https://testingbot.com/members/user/edit). You'll find: - **`key` string:** Your public API client key. Safe to share between your CI and the TestingBot grid; not a secret on its own. - **`secret` string sensitive:** Treat this like a password. Never commit it to git, never paste it into screenshots, never embed it in client-side code. Use environment variables or a secret manager in CI. ### 2 · Send credentials on every request Pass the credentials as the standard `Authorization` header. Most HTTP clients accept a `user:password` tuple and handle the Base64 encoding for you — examples on the right. ### 3 · Verify it works Hit [GET /v1/user](https://testingbot.com#user). A JSON response with your name and plan means auth is set up correctly. A `401 Unauthorized` means the key or secret is wrong. **Best practice.** Store credentials in environment variables — `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` — and inject them into your CI/CD pipeline as secrets. Rotate the secret immediately if it leaks (Account → Reset API credentials). `Authentication Examples` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Authenticated request ```bash $ curl https://api.testingbot.com/v1/user \ -u "$TESTINGBOT_KEY:$TESTINGBOT_SECRET" ``` ```csharp using TestingBot.Api; // reads TESTINGBOT_KEY / TESTINGBOT_SECRET from the environment using var client = TestingBotClient.FromEnvironment(); var user = await client.User.GetAsync(); ``` ```ruby require 'testingbot' api = TestingBot::Api.new( ENV['TESTINGBOT_KEY'], ENV['TESTINGBOT_SECRET'] ) api.get_user_info ``` ```python import os, testingbotclient tb = testingbotclient.TestingBotClient( os.environ['TESTINGBOT_KEY'], os.environ['TESTINGBOT_SECRET'] ) tb.user.get_user_information() ``` ```php $api = new TestingBot\TestingBotAPI( getenv('TESTINGBOT_KEY'), getenv('TESTINGBOT_SECRET') ); $api->getUserInfo(); ``` ```java TestingbotREST restApi = new TestingbotREST( System.getenv("TESTINGBOT_KEY"), System.getenv("TESTINGBOT_SECRET") ); TestingbotUser user = restApi.getUserInfo(); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: process.env.TESTINGBOT_KEY, api_secret: process.env.TESTINGBOT_SECRET }); const userInfo = await api.getUserInfo(); ``` ## Reducing response size Any endpoint accepts an optional `omit` query parameter to drop fields you don't need from the JSON response. Pass a comma-separated list of field names. This is handy for trimming large payloads, for example skipping the embedded `report` and `report_xml` on App Automate run details. - **`omit` string:** Comma-separated field names to exclude. Matching is by field name and applies anywhere it appears in the response, including inside `data` list items and nested objects. Unknown names are ignored. **Note.** `omit` never affects whether a request succeeds, only the shape of the body. Omitting nothing returns the full, default response. `omit example` Skip heavy fields on a run ```bash $ curl "https://api.testingbot.com/v1/app-automate/maestro/12/45?omit=report,report_xml" \ -u "$TESTINGBOT_KEY:$TESTINGBOT_SECRET" ``` Response ```json { "id": 45, "status": "DONE", "success": true, "created_at": "2026-06-01T10:00:00.000Z" } ``` ## Rate limits Requests are counted in a rolling five-minute window. Authenticated requests count against your **API key** , so one busy CI job cannot spend a colleague's budget, and a key that leaks is capped rather than free. - **`2000 requests / 5 minutes` per API key:** Roughly 6 requests per second sustained. Applies to every authenticated endpoint. - **`300 requests / 5 minutes` per IP:** For requests sent without credentials, which can only reach the public endpoints such as [GET /v1/browsers](https://testingbot.com/support/api/devices#browsers). - **`10000 requests / 5 minutes` per IP, all calls:** An overall ceiling on everything sent from one IP address, whichever API keys it uses. Set well above a single key's allowance, so several keys behind one CI egress address fit comfortably. Every [tunnel endpoint](https://testingbot.com/support/api/tunnel) is exempt. Tunnels are infrastructure rather than a data surface: the client polls while a tunnel boots, and the tunnel VM itself reports readiness, so a limit there would stall provisioning rather than stop abuse. [Sending a test webhook delivery](https://testingbot.com/support/api/webhooks#testwebhook) has its own, much tighter allowance of **20 per 5 minutes per API key** , on top of the limit above. Each call makes TestingBot perform an outbound request to a URL you choose, and verifying a webhook is a setup-time action rather than something to do in bulk. Every API response carries your current usage, so you can slow down before you are cut off rather than after: - **`X-RateLimit-Limit` integer:** Requests allowed in the current window. - **`X-RateLimit-Remaining` integer:** Requests left in the current window. - **`X-RateLimit-Reset` integer:** UNIX timestamp at which the window resets and the allowance returns to full. **Going over.** Requests past the limit return `429 Too Many Requests` with a `Retry-After` header giving the seconds to wait. The body is JSON in the usual shape, so an SDK surfaces it as an API error rather than a parse failure. Retrying after `Retry-After` is always safe. `rate limit headers` On every response ``` X-RateLimit-Limit: 2000 X-RateLimit-Remaining: 1994 X-RateLimit-Reset: 1785736800 ``` Over the limit ```json { "error": "429 Too Many Requests. Retry in 300 seconds." } ``` --- URL: https://testingbot.com/support/api/devices # Browsers & Devices Discover which browsers, operating systems and real devices you can run on, and look up the IP ranges to allowlist. - **Endpoint:** api.testingbot.com - **Version:** v1 - **Format:** JSON - **Auth:** [HTTP Basic](https://testingbot.com/support/api#authentication) GET `/v1/browsers` ## List supported browsers Returns every browser environment available on the TestingBot grid: name, version, platform, and the `browser_id` used when attaching browsers to Codeless tests. No authentication required. ### Arguments - **`type` string:** Filter by automation protocol: "webdriver" (default) or "rc" (legacy Selenium RC). ### Response fields - **`selenium_name` string:** Capability value to send as `browserName` in WebDriver. - **`name` string:** Human-readable browser identifier (e.g. "firefox", "iexplore"). - **`version` string:** Version as a string, not a number: it may carry minor/patch components (e.g. "11.4"). Omitted, along with `long_version`, when the environment has no version set. - **`long_version` string:** Full version string (e.g. "121.0.6167.184"). Omitted when the environment has no version set. - **`platform` string:** Operating system (e.g. "WINDOWS", "MAC", "LINUX"). - **`browser_id` integer:** Unique TestingBot browser ID used to attach browsers to Codeless tests. GET `/v1/browsers` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash curl "https://api.testingbot.com/v1/browsers" ``` ```csharp var client = new TestingBotClient(key, secret); var browsers = await client.Browsers.ListAsync(); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_browsers ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.information.get_browsers() ``` ```php $api = new TestingBot\TestingBotAPI($key, $secret); $api->getBrowsers(); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); ArrayList browsers = restApi.getBrowsers(); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const browsers = await api.getBrowsers(); ``` Response ```json [ {"selenium_name": "IE8", "name": "iexplore", "version": "8", "platform": "WINDOWS", "browser_id": 1, "long_version": "8"}, {"selenium_name": "FF7", "name": "firefox", "version": "7", "platform": "WINDOWS", "browser_id": 22, "long_version": "7"} ] ``` GET `/v1/devices` ## List physical mobile devices Returns every real Android and iOS device in the TestingBot grid, with an `available` flag indicating whether it can be acquired right now. Filter by `platform` for a single OS family. ### Arguments - **`platform` string:** Filter to a single mobile OS family. Case-insensitive. - **`web` boolean:** Internal flag used by the dashboard — returns extra UI-only fields. ### Response fields - **`id` integer:** Unique numeric device ID. - **`name` string:** Marketing name (e.g. "iPhone 15 Pro"). - **`model_number` string:** Manufacturer model number (e.g. "SM-G991B"). - **`platform_name` string:** Mobile OS family (e.g. "iOS", "Android"). - **`version` string:** OS version (e.g. "17.4", "14"). - **`resolution` string:** Display resolution as `x` (e.g. "1170x2532"). - **`width` integer:** Display width in pixels, parsed from `resolution`. - **`height` integer:** Display height in pixels, parsed from `resolution`. - **`cpu` string:** CPU architecture. - **`available` boolean:** Whether the device is free for this account to acquire right now. - **`multiple_browsers` array of object:** Selectable browsers on this device. Only present on devices that offer more than one. GET `/v1/devices` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl -u key:secret "https://api.testingbot.com/v1/devices" ``` ```csharp var client = new TestingBotClient(key, secret); var devices = await client.Devices.ListAsync(); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_devices ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.information.get_devices() ``` ```php $api = new TestingBot\TestingBotAPI($key, $secret); $api->getDevices(); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); List devices = restApi.getDevices(0, 100); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const devices = await api.getDevices(); ``` Response ```json [ { "id": 1, "available": true, "name": "Galaxy S8", "model_number": "SM-G950U1", "platform_name": "Android", "version": "9.0", "resolution": "1080x2220", "multiple_browsers": [ { "id": 1727, "name": "chrome", "version": "103" }, { "id": 2133, "name": "firefox", "version": "98.2" }, { "id": 2953, "name": "samsung", "version": "13.2.1.70" }, { "id": 2954, "name": "opera", "version": "67.1" } ] }, { "id": 5, "available": false, "name": "iPhone XR", "platform_name": "iOS", "version": "16.3", "resolution": "828x1792" } ] ``` GET `/v1/devices/available` ## List currently available devices Subset of /v1/devices filtered to devices the authenticated account can acquire right now (not in use by another customer, not under maintenance). ### Response fields - **`id` integer:** Unique numeric device ID. - **`name` string:** Marketing name (e.g. "iPhone 15 Pro"). - **`model_number` string:** Manufacturer model number (e.g. "SM-G991B"). - **`platform_name` string:** Mobile OS family (e.g. "iOS", "Android"). - **`version` string:** OS version (e.g. "17.4", "14"). - **`resolution` string:** Display resolution as `x` (e.g. "1170x2532"). - **`width` integer:** Display width in pixels, parsed from `resolution`. - **`height` integer:** Display height in pixels, parsed from `resolution`. - **`cpu` string:** CPU architecture. - **`available` boolean:** Whether the device is free for this account to acquire right now. - **`multiple_browsers` array of object:** Selectable browsers on this device. Only present on devices that offer more than one. GET `/v1/devices/available` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl -u key:secret "https://api.testingbot.com/v1/devices/available" ``` ```csharp var client = new TestingBotClient(key, secret); var devices = await client.Devices.ListAvailableAsync(); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_available_devices ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.information.get_available_devices() ``` ```php $api = new TestingBot\TestingBotAPI($key, $secret); $api->getAvailableDevices(); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); List devices = restApi.getAvailableDevices(); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const devices = await api.getAvailableDevices(); ``` Response ```json [ { "id": 1, "available": true, "name": "Galaxy S8", "platform_name": "Android", "version": "9.0", "resolution": "1080x2220" } ] ``` GET `/v1/devices/:id` ## Get a specific device Returns the spec and current availability for a single device by numeric ID. ### Arguments - **`id` integer required:** Numeric device ID. ### Response fields - **`id` integer:** Unique numeric device ID. - **`name` string:** Marketing name (e.g. "iPhone 15 Pro"). - **`model_number` string:** Manufacturer model number (e.g. "SM-G991B"). - **`platform_name` string:** Mobile OS family (e.g. "iOS", "Android"). - **`version` string:** OS version (e.g. "17.4", "14"). - **`resolution` string:** Display resolution as `x` (e.g. "1170x2532"). - **`width` integer:** Display width in pixels, parsed from `resolution`. - **`height` integer:** Display height in pixels, parsed from `resolution`. - **`cpu` string:** CPU architecture. - **`available` boolean:** Whether the device is free for this account to acquire right now. - **`multiple_browsers` array of object:** Selectable browsers on this device. Only present on devices that offer more than one. GET `/v1/devices/:id` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/devices/:id" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var device = await client.Devices.GetAsync(deviceId); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_device(device_id) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.information.get_device(device_id) ``` ```php $api = new TestingBot\TestingBotAPI($key, $secret); $api->getDevice($deviceId); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingbotDevice device = restApi.getDevice(deviceId); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const device = await api.getDevice(deviceId); ``` Response ```json { "id": 12, "available": true, "name": "Galaxy S20", "platform_name": "Android", "version": "10.0", "resolution": "1440x3200", "multiple_browsers": [ { "id": 2598, "name": "chrome", "version": "101" }, { "id": 2919, "name": "firefox", "version": "98.2" }, { "id": 2957, "name": "samsung", "version": "13.2.1.70" } ] } ``` GET `/v1/configuration/ip-ranges` ## TestingBot IP ranges for firewall whitelisting Returns the up-to-date list of public IPv4 addresses used by TestingBot test machines. Allow these in your firewall when running tests against staging or production behind corporate VPN/network rules. The response is a flat JSON array of IPv4 strings (no CIDR ranges, no metadata) — re-fetch periodically since the pool changes as we scale capacity. Unauthenticated; no `key`/`secret` required. GET `/v1/configuration/ip-ranges` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/configuration/ip-ranges" ``` ```csharp var client = new TestingBotClient(key, secret); var ranges = await client.Configuration.GetIpRangesAsync(); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_ip_ranges ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.configuration.get_ip_ranges() ``` ```php $client = new TestingBot\Client($key, $secret); $ranges = $client->configuration()->ipRanges(); // no authentication required ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); JsonElement ipRanges = restApi.getIpRanges(); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const ipRanges = await api.getIpRanges(); ``` Response ```json [ "109.68.162.161", "109.68.162.165", "78.20.187.211", "162.55.135.22", "95.217.83.184", "95.216.39.110", "95.216.229.109", "142.132.137.7", "195.201.167.43", "162.55.24.100", "185.223.133.13" ] ``` [Next User & Team Management](https://testingbot.com/support/api/user) --- URL: https://testingbot.com/support/api/user # User & Team Management Your own account and credentials, plus everything needed to provision team members and service accounts from a script. - **Endpoint:** api.testingbot.com - **Version:** v1 - **Format:** JSON - **Auth:** [HTTP Basic](https://testingbot.com/support/api#authentication) GET `/v1/user` ## Get your user info Returns the data associated with the authenticated account: name, plan, billing details, remaining credits, and concurrency caps. ### Response fields - **`id` integer:** Unique numeric user ID. Also the path param for team endpoints (e.g. /users/:id) and the parent\_id reference on team members. - **`first_name` string:** Given name on the account. - **`last_name` string:** Family name on the account. - **`seconds` integer:** Remaining test seconds (credit balance) on the account; -1 means unlimited (e.g. Live plans). - **`last_login` timestamp:** ISO-8601 timestamp of the most recent dashboard login. - **`plan` string:** Subscription plan name (e.g. "Free Trial", "Live - 5 Sessions - Unlimited", "Automated Pro ..."). For a sub-account this is the team owner's plan. - **`max_concurrent` integer:** Maximum number of parallel VM-based sessions allowed by the plan. For a sub-account this is the team owner's cap. - **`max_concurrent_mobile` integer:** Maximum number of parallel physical device sessions allowed by the plan. For a sub-account this is the team owner's cap. - **`company` string:** Optional company name for billing. - **`street` string:** Billing address line. - **`city` string:** Billing city. - **`country` string:** Billing country. - **`vat` string:** VAT number (EU only). - **`read_only` boolean:** Whether this account has read-only access within the team. - **`roles` array of string:** Roles held by the account: "admin" and/or "owner". - **`current_vm_concurrency` integer:** VM-based sessions this account is running right now. - **`current_physical_concurrency` integer:** Physical-device sessions this account is running right now. GET `/v1/user` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/user" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var user = await client.User.GetAsync(); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_user_info ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.user.get_user_information() ``` ```php $api = new TestingBot\TestingBotAPI($key, $secret); $api->getUserInfo(); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingbotUser user = restApi.getUserInfo(); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const userInfo = await api.getUserInfo(); ``` Response ```json { "first_name": "Steven", "last_name": "King", "seconds": 17745, "plan": "small", "max_concurrent": 10, "max_concurrent_mobile": 2, "company": "companyName", "street": "companyStreet", "city": "companyCity", "country": "companyCountry", "vat": "ifInEurope" } ``` PUT `/v1/user` ## Update your user info Updates the authenticated account. Only `first_name` and `last_name` are mutable through this endpoint; other profile fields require the dashboard. ### Arguments - **`user[first_name]` string:** New given name. - **`user[last_name]` string:** New family name. ### Response fields - **`success` boolean:** Whether the update was applied. - **`user` object:** Updated user object. Only present when `success` is true. - **`errors` object:** Validation errors keyed by field name. Only present when `success` is false. PUT `/v1/user` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/user" \ -X PUT \ -d "user[first_name]=new" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); await client.User.UpdateAsync(new UserUpdate { FirstName = "Bruno", LastName = "Mars" }); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.update_user_info({ "first_name" => 'new' }) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.user.update_user_information({'user[first_name]': 'new'}) ``` ```php $api = new TestingBot\TestingBotAPI($key, $secret); $api->updateUserInfo(array('first_name' => 'new')); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingbotUser user = restApi.updateUserInfo(TestingBotUser); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const newUserData = { first_name: 'new' }; await api.updateUserInfo(newUserData); ``` Response ```json { "success": true, "user": { "first_name": "new", "last_name": "King", "seconds": 17745, "last_login": "2011-08-06T16:47:04Z" } } ``` GET `/v1/user/keys` ## Get your API key and secret Returns the API client key and secret for the authenticated account. Useful for confirming the credentials currently in use, but never embed this response in client-side code. ### Response fields - **`key` string:** API client key. - **`secret` string:** API client secret. Treat as a password: never embed it in client-side code. GET `/v1/user/keys` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/user/keys" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var keys = await client.User.GetKeysAsync(); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_user_keys ``` ```php $client = new TestingBot\Client($key, $secret); $keys = $client->user()->keys(); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); JsonElement keys = restApi.getUserKeys(); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const { key, secret } = await api.getUserKeys(); ``` GET `/v1/team-management` ## Get team concurrency info Returns allowed vs current concurrent session counts for the team across both VMs and physical mobile devices, plus how many sessions are waiting for a free slot. Poll this before fanning out a parallel build. Counts are team-wide and come from the Hub, so they match what the dashboard shows. ### Response fields - **`concurrency` object:** Allowed vs current concurrency caps for the team, split in VM and physical device sessions. GET `/v1/team-management` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/team-management" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var concurrency = await client.Team.GetConcurrencyAsync(); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_team ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.team.get_concurrency() ``` ```php $client = new TestingBot\Client($key, $secret); $team = $client->teamManagement()->get(); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingbotTeam team = restApi.getTeam(); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const teamInfo = await api.getTeam(); ``` Response ```json { "concurrency": { "allowed": { "vms": 10, "physical": 2 }, "current": { "vms": 10, "physical": 1 }, "queued": { "vms": 3, "physical": 0 } } } ``` GET `/v1/team-management/users` ## List users in your team Paginated list of all sub-accounts in the team. Requires admin role on the calling account. ### Arguments - **`offset` integer:** Skip this many users from the start of the result set. - **`count` integer max=`500`:** Number of users to return . ### Response fields - **`data` array of team member objects:** Team member accounts. - **`meta` meta object:** — GET `/v1/team-management/users` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/team-management/users" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var members = await client.Team.ListUsersAsync(); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_users_in_team(0, 10) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.team.get_users(offset=0, limit=10) ``` ```php $client = new TestingBot\Client($key, $secret); $members = $client->teamManagement()->listUsers(0, 10); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingbotTeamMemberCollection members = restApi.getTeamMembers(); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const users = await api.getUsersInTeam(); ``` Response ```json { "data": [ { "id": 337, "first_name": "test", "last_name": "user", "seconds": 12000, "plan": "Free Trial", "max_concurrent": 2, "max_concurrent_mobile": 2 } ], "meta": { "offset": 0, "count": 1, "total": 1 } } ``` GET `/v1/team-management/users/:id` ## Get a specific team user Returns the user record for a specific team member. Admins can fetch any team member; non-admins can only fetch themselves. ### Arguments - **`id` integer required:** Numeric ID of the team user to fetch. ### Response fields - **`id` integer:** Unique numeric user ID. Also the path param for team endpoints (e.g. /users/:id) and the parent\_id reference on team members. - **`first_name` string:** Given name on the account. - **`last_name` string:** Family name on the account. - **`seconds` integer:** Remaining test seconds (credit balance) on the account; -1 means unlimited (e.g. Live plans). - **`last_login` timestamp:** ISO-8601 timestamp of the most recent dashboard login. - **`plan` string:** Subscription plan name (e.g. "Free Trial", "Live - 5 Sessions - Unlimited", "Automated Pro ..."). For a sub-account this is the team owner's plan. - **`max_concurrent` integer:** Maximum number of parallel VM-based sessions allowed by the plan. For a sub-account this is the team owner's cap. - **`max_concurrent_mobile` integer:** Maximum number of parallel physical device sessions allowed by the plan. For a sub-account this is the team owner's cap. - **`company` string:** Optional company name for billing. - **`street` string:** Billing address line. - **`city` string:** Billing city. - **`country` string:** Billing country. - **`vat` string:** VAT number (EU only). - **`read_only` boolean:** Whether this account has read-only access within the team. - **`roles` array of string:** Roles held by the account: "admin" and/or "owner". - **`current_vm_concurrency` integer:** VM-based sessions this account is running right now. - **`current_physical_concurrency` integer:** Physical-device sessions this account is running right now. GET `/v1/team-management/users/:id` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/team-management/users/:id" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var member = await client.Team.GetUserAsync(userId); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_user_in_team(user_id) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.team.get_user(user_id) ``` ```php $client = new TestingBot\Client($key, $secret); $member = $client->teamManagement()->getUser($userId); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingbotTeamMember member = restApi.getTeamMember(userId); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const user = await api.getUserFromTeam(userId); ``` Response ```json { "id": 337, "first_name": "test", "last_name": "user", "seconds": 12000, "plan": "Free Trial", "max_concurrent": 2, "max_concurrent_mobile": 2 } ``` POST `/v1/team-management/users` ## Create a user in your team Provisions a new sub-account inside the team, copying the caller's subscription level and assigning a slice of the credit pool. Requires an upgraded plan and admin role. ### Arguments - **`email` string required:** Email address for the new account; must be unique across TestingBot. - **`password` string required:** Initial password for the new user. - **`first_name` string required:** New user's given name. - **`last_name` string required:** New user's family name. - **`concurrency` integer:** Max parallel VM sessions (≤ team owner's `maxParallel`). - **`concurrencyPhysical` integer:** Max parallel physical-device sessions (≤ team owner's `maxParallelDevice`). POST `/v1/team-management/users` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl -X POST -u key:secret \ https://api.testingbot.com/v1/team-management/users \ --header 'Content-Type: application/json' \ --data-raw '{ "first_name": "Bruno", "last_name": "Mars", "email": "bmars@example.com", "password": "$bmaRs*RULES" }' ``` ```csharp var client = new TestingBotClient(key, secret); var member = await client.Team.CreateUserAsync(new TeamMemberCreate { Email = "bmars@example.com", Password = "$bmaRs*RULES", FirstName = "Bruno", LastName = "Mars" }); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.create_user_in_team({ first_name: 'Bruno', last_name: 'Mars', email: 'bmars@example.com', password: '$bmaRs*RULES' }) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.team.create_user(email='bmars@example.com', password='$bmaRs*RULES', first_name='Bruno', last_name='Mars') ``` ```php $client = new TestingBot\Client($key, $secret); $member = $client->teamManagement()->createUser([ 'email' => 'bmars@example.com', 'password' => '$bmaRs*RULES', 'first_name' => 'Bruno', 'last_name' => 'Mars', ]); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); Map params = new HashMap<>(); params.put("first_name", "Bruno"); params.put("last_name", "Mars"); params.put("email", "bmars@example.com"); params.put("password", "$bmaRs*RULES"); TestingbotTeamMember member = restApi.createTeamMember(params); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const userData = { first_name: 'John', last_name: 'Doe', email: 'john@example.com', password: 'StrongPassword!' }; const result = await api.createUserInTeam(userData); ``` Response ```json { "id": 337, "first_name": "Bruno", "last_name": "Mars", "seconds": 12000, "plan": "Free Trial", "max_concurrent": 2, "max_concurrent_mobile": 2 } ``` PUT `/v1/team-management/users/:id` ## Update a user in your team Updates a team user's profile and credit allocation. Credit fields cannot exceed the team owner's remaining balance. Members can change their own email or password only, and must send `current_password` to do so. ### Arguments - **`id` integer required:** Numeric ID of the team user to update. - **`first_name` string:** New given name. - **`last_name` string:** New family name. - **`email` string:** New email address. - **`password` string:** New password. - **`current_password` string:** Your current password. Required when changing your own email or password. - **`credits` integer:** Allocated VM credit seconds (≤ team owner's remaining VM credits). - **`device_credits` integer:** Allocated physical-device credit seconds (≤ team owner's remaining device credits). - **`concurrency` integer:** Max parallel VM sessions for this user. - **`concurrencyPhysical` integer:** Max parallel physical-device sessions for this user. ### Response fields - **`id` integer:** Unique numeric user ID. Also the path param for team endpoints (e.g. /users/:id) and the parent\_id reference on team members. - **`first_name` string:** Given name on the account. - **`last_name` string:** Family name on the account. - **`seconds` integer:** Remaining test seconds (credit balance) on the account; -1 means unlimited (e.g. Live plans). - **`last_login` timestamp:** ISO-8601 timestamp of the most recent dashboard login. - **`plan` string:** Subscription plan name (e.g. "Free Trial", "Live - 5 Sessions - Unlimited", "Automated Pro ..."). For a sub-account this is the team owner's plan. - **`max_concurrent` integer:** Maximum number of parallel VM-based sessions allowed by the plan. For a sub-account this is the team owner's cap. - **`max_concurrent_mobile` integer:** Maximum number of parallel physical device sessions allowed by the plan. For a sub-account this is the team owner's cap. - **`company` string:** Optional company name for billing. - **`street` string:** Billing address line. - **`city` string:** Billing city. - **`country` string:** Billing country. - **`vat` string:** VAT number (EU only). - **`read_only` boolean:** Whether this account has read-only access within the team. - **`roles` array of string:** Roles held by the account: "admin" and/or "owner". - **`current_vm_concurrency` integer:** VM-based sessions this account is running right now. - **`current_physical_concurrency` integer:** Physical-device sessions this account is running right now. PUT `/v1/team-management/users/:id` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl -X PUT -u key:secret \ https://api.testingbot.com/v1/team-management/users/:id \ --header 'Content-Type: application/json' \ --data-raw '{ "first_name": "Bruno", "last_name": "Mars" }' ``` ```csharp var client = new TestingBotClient(key, secret); var member = await client.Team.UpdateUserAsync(userId, new TeamMemberUpdate { FirstName = "Bruno", LastName = "Mars" }); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.update_user_in_team(user_id, { first_name: 'Bruno', last_name: 'Mars' }) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.team.update_user(user_id, first_name='Bruno', last_name='Mars') ``` ```php $client = new TestingBot\Client($key, $secret); $client->teamManagement()->updateUser($userId, ['first_name' => 'Bruno']); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); Map params = new HashMap<>(); params.put("first_name", "Bruno"); params.put("last_name", "Mars"); TestingbotTeamMember member = restApi.updateTeamMember(userId, params); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const result = await api.updateUserInTeam(userId, { first_name: 'Jane', last_name: 'Smith' }); ``` Response ```json { "id": 337, "first_name": "Bruno", "last_name": "Mars", "seconds": 12000, "plan": "Free Trial" } ``` POST `/v1/team-management/users/:id/reset-keys` ## Reset credentials for a team user Rotates both the API key and secret for a team user. The old credentials stop working immediately, so any deployed automation using them must be updated. ### Arguments - **`id` integer required:** Numeric ID of the team user whose credentials to rotate. POST `/v1/team-management/users/:id/reset-keys` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl -X POST -u key:secret \ https://api.testingbot.com/v1/team-management/users/:id/reset-keys ``` ```csharp var client = new TestingBotClient(key, secret); var reset = await client.Team.ResetUserKeysAsync(userId); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.reset_credentials(user_id) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.team.reset_keys(user_id) ``` ```php $client = new TestingBot\Client($key, $secret); $credentials = $client->teamManagement()->resetUserKeys($userId); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingbotTeamCredentialReset reset = restApi.resetTeamMemberKeys(userId); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const result = await api.resetCredentials(userId); ``` Response ```json { "success": true, "client_key": "d69e0800xc32ed2f059b7a630ee255b" } ``` GET `/v1/team-management/service-accounts` ## List service accounts Returns the team's API-only service accounts, each with its own key and secret. Service accounts run tests on the grid and REST API but cannot sign in to the dashboard. Admin only. ### Arguments - **`offset` integer:** Skip this many service accounts from the start of the result set. - **`count` integer:** Number of service accounts to return. ### Response fields - **`data` array of service account objects:** Service accounts for this team. - **`meta` meta object:** — GET `/v1/team-management/service-accounts` [cURL](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/team-management/service-accounts" \ -u key:secret ``` Response ```json { "data": [ { "id": 412, "identifier": "ci-runner", "key": "a1b2c3d4e5f60718293a4b5c6d7e8f90", "secret": "f0e9d8c7b6a5948372615f4e3d2c1b0a", "created_at": "2026-06-05T10:08:35Z" } ], "meta": { "offset": 0, "count": 10, "total": 1 } } ``` GET `/v1/team-management/service-accounts/:id` ## Get a service account Returns a single service account, including its key and secret. Admin only. ### Arguments - **`id` integer required:** Numeric ID of the service account. ### Response fields - **`id` integer:** Unique numeric service-account ID. - **`identifier` string:** Human-friendly label shown on the dashboard and test detail pages. - **`key` string:** TestingBot API key (username) for this service account. - **`secret` string:** TestingBot API secret for this service account. Store it as a CI secret. - **`created_at` timestamp:** When the service account was created. GET `/v1/team-management/service-accounts/:id` [cURL](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/team-management/service-accounts/:id" \ -u key:secret ``` Response ```json { "id": 412, "identifier": "ci-runner", "key": "a1b2c3d4e5f60718293a4b5c6d7e8f90", "secret": "f0e9d8c7b6a5948372615f4e3d2c1b0a", "created_at": "2026-06-05T10:08:35Z" } ``` POST `/v1/team-management/service-accounts` ## Create a service account Creates a new API-only service account with its own generated key and secret. The identifier is a label (min 3 characters, unique within the team) shown next to the account's tests on the dashboard. Admin only. ### Arguments - **`identifier` string required min=`3 characters`:** Human-friendly label for the service account (min 3 characters, unique within the team). POST `/v1/team-management/service-accounts` [cURL](https://testingbot.com#) Request ```bash $ curl -X POST -u key:secret \ https://api.testingbot.com/v1/team-management/service-accounts \ --header 'Content-Type: application/json' \ --data-raw '{ "identifier": "ci-runner" }' ``` Response ```json { "id": 412, "identifier": "ci-runner", "key": "a1b2c3d4e5f60718293a4b5c6d7e8f90", "secret": "f0e9d8c7b6a5948372615f4e3d2c1b0a", "created_at": "2026-06-05T10:08:35Z" } ``` PUT `/v1/team-management/service-accounts/:id` ## Update a service account Renames a service account by changing its identifier. The key and secret are unchanged. Admin only. ### Arguments - **`id` integer required:** Numeric ID of the service account to update. - **`identifier` string required min=`3 characters`:** New identifier (min 3 characters, unique within the team). ### Response fields - **`id` integer:** Unique numeric service-account ID. - **`identifier` string:** Human-friendly label shown on the dashboard and test detail pages. - **`key` string:** TestingBot API key (username) for this service account. - **`secret` string:** TestingBot API secret for this service account. Store it as a CI secret. - **`created_at` timestamp:** When the service account was created. PUT `/v1/team-management/service-accounts/:id` [cURL](https://testingbot.com#) Request ```bash $ curl -X PUT -u key:secret \ https://api.testingbot.com/v1/team-management/service-accounts/:id \ --header 'Content-Type: application/json' \ --data-raw '{ "identifier": "nightly-regression" }' ``` Response ```json { "id": 412, "identifier": "nightly-regression", "key": "a1b2c3d4e5f60718293a4b5c6d7e8f90", "secret": "f0e9d8c7b6a5948372615f4e3d2c1b0a", "created_at": "2026-06-05T10:08:35Z" } ``` POST `/v1/team-management/service-accounts/:id/rotate` ## Rotate a service account access key Generates a fresh key and secret for the service account. The previous key and secret stop working immediately, so update any automation using them. Admin only. ### Arguments - **`id` integer required:** Numeric ID of the service account whose key and secret to rotate. POST `/v1/team-management/service-accounts/:id/rotate` [cURL](https://testingbot.com#) Request ```bash $ curl -X POST -u key:secret \ https://api.testingbot.com/v1/team-management/service-accounts/:id/rotate ``` Response ```json { "id": 412, "identifier": "ci-runner", "key": "9f8e7d6c5b4a3i2h1g0f9e8d7c6b5a40", "secret": "40a5b6c7d8e9f0g1h2i3a4b5c6d7e8f9", "created_at": "2026-06-05T10:08:35Z" } ``` DELETE `/v1/team-management/service-accounts/:id` ## Delete a service account Permanently deletes the service account. Its key and secret stop working immediately. Tests it already ran stay in your history, still labelled with its identifier. Admin only. ### Arguments - **`id` integer required:** Numeric ID of the service account to delete. DELETE `/v1/team-management/service-accounts/:id` [cURL](https://testingbot.com#) Request ```bash $ curl -X DELETE -u key:secret \ https://api.testingbot.com/v1/team-management/service-accounts/:id ``` Response ```json { "success": true, "message": "Service account \"ci-runner\" deleted." } ``` GET `/v1/team-management/users/{id}/client-key` ## Get a team user's client key Admin-only endpoint that returns the API client key for a specific team user. Used by tooling that impersonates team members in CI; the secret is not exposed here — use reset-keys to rotate. ### Arguments - **`id` integer required:** Numeric ID of the team user whose key to fetch. ### Response fields - **`client_key` string:** The API client key. The matching secret is never returned here; rotate both with reset-keys. GET `/v1/team-management/users/{id}/client-key` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/team-management/users/{id}/client-key" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var clientKey = await client.Team.GetUserClientKeyAsync(userId); ``` ```php $client = new TestingBot\Client($key, $secret); $clientKey = $client->teamManagement()->getUserClientKey($userId); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); JsonElement clientKey = restApi.getTeamMemberClientKey(userId); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const { client_key } = await api.getUserClientKey(userId); ``` [Previous Browsers & Devices](https://testingbot.com/support/api/devices) [Next Tests & Builds](https://testingbot.com/support/api/tests) --- URL: https://testingbot.com/support/api/tests # Tests & Builds Fetch test results, report pass/fail state back from your CI, and work with the builds that group them. - **Endpoint:** api.testingbot.com - **Version:** v1 - **Format:** JSON - **Auth:** [HTTP Basic](https://testingbot.com/support/api#authentication) **Changed August 2026:** the `browser` field is now a stable identifier without the version (for example `googlechrome` rather than `Chrome 150`), so it is safe to match on. The human-readable label moved to the new `browser_display_name` field, and `browser_long_version` carries the full version string when known. Integrations that matched on the old combined label should match on `browser` plus `browser_version` instead. GET `/v1/tests` ## List your tests Paginated list of every test session belonging to the authenticated account, newest first. Filter by `browser_id`, `group`, or `build` to narrow the result. Use `since` to fetch only tests updated after a UNIX timestamp (poll-friendly). ### Arguments - **`offset` integer:** Skip this many tests from the start of the result set. - **`count` integer max=`500`:** Number of tests to return . - **`since` integer:** UNIX timestamp; return only tests updated at or after this time. - **`browser_id` integer:** Filter to tests that ran on this browser\_id (from /v1/browsers). - **`group` string:** Filter to tests tagged with this group name. - **`build` string:** Filter to tests in this build (matches `capabilities.build`). - **`skip_fields` string:** Comma-separated fields to omit (logs, thumbs). ### Response fields - **`data` array of test case objects:** Test sessions for this page. - **`meta` meta object:** — - **`realtime` object:** ActionCable channel details for live updates. Only present on the `since`-polling form of this endpoint. GET `/v1/tests` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/tests?offset=0&count=10" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var tests = await client.Tests.ListAsync(); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_tests(0, 10) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.tests.get_tests(offset=0, limit=10) ``` ```php $api = new TestingBot\TestingBotAPI($key, $secret); $api->getJobs(0, 10); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingbotTestCollection tests = restApi.getTests(0, 10); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const tests = await api.getTests({ offset, limit }); ``` Response ```json { "data": [ { "created_at": "2011-07-30T23:21:23Z", "completed_at": "2011-07-30T23:22:44Z", "id": 3, "name": "MyTest::testTitle", "session_id": "f7903f9e93e74fe1b0e924bf9d2ce9fc", "status_message": "Failed asserting that 1 equals 0.", "status_id": 0, "success": false, "browser": "iexplore", "browser_version": "8", "browser_long_version": "8.0.7601", "browser_display_name": "IE8", "os": "WINDOWS", "duration": 13, "build": "buildid", "video": "https://s3.amazonaws.com/rectestingbot/sample.mp4", "groups": ["testingbot.com"] } ], "meta": { "offset": 0, "count": 10, "total": 789 } } ``` GET `/v1/tests/:id` ## Get a specific test Returns the full detail record for a single test session: status, environment (browser/OS or device/platform), assets (video, logs, screenshots), groups, build, and duration. Accepts either the numeric test ID or the WebDriver session\_id. ### Arguments - **`id` string required:** Numeric test ID or WebDriver session\_id (UUID). - **`skip_fields` string:** Comma-separated fields to omit from the response (logs, thumbs, visual\_run, groups). ### Response fields - **`id` integer:** Unique numeric test ID. - **`session_id` string:** Selenium / WebDriver session ID (UUID). - **`name` string:** Human-readable test name set via capabilities or `update_test`. - **`state` string:** Lifecycle state (RUNNING, COMPLETE, TIMEOUT, …). - **`success` boolean:** Whether the test passed. - **`status_id` integer:** Numeric status code (0=fail, 1=pass, 2=unknown). - **`unknown` boolean:** Convenience flag, true when `status_id` is 2 (the test finished without reporting a pass/fail). - **`status_message` string:** Failure reason or arbitrary status set via `test[status_message]`. - **`created_at` timestamp:** When the test session started. - **`completed_at` timestamp:** When the session ended; null while running. - **`duration` integer:** Total run time in seconds. - **`browser` string:** Browser identifier, without the version (e.g. "googlechrome", "iexplore", "firefox", "safari"). Stable across versions, so it is the field to match on. - **`browser_version` string:** Browser version as a string, not a number: it may carry minor/patch components (e.g. "150", "11.4"), and is "0" for environments with no meaningful version. - **`browser_long_version` string:** Full version string when known (e.g. "121.0.6167.184"); null otherwise. - **`browser_display_name` string:** Human-readable label for UIs (e.g. "Chrome 150", "IE11", "Safari 11"), with the device and OS appended on mobile. Presentation only: the format is not stable, so build UI strings from it but never match on it. - **`os` string:** OS (e.g. "WINDOWS", "MAC"). - **`device_name` string:** Mobile device name when the session ran on a physical device; null otherwise. - **`platform_name` string:** Mobile OS name; null on desktop. - **`build` string:** Build identifier (free-form string set via capabilities). - **`groups` array of string:** Tag/group names attached to the test. - **`video` string:** Signed S3 URL to the recorded video, or false if video was disabled. - **`thumbs` array of string:** Signed S3 URLs to screenshot thumbnails. - **`logs` object:** Map of log names → signed S3 URLs (selenium, browser, …). - **`assets_available` boolean:** Whether assets (video, logs, screenshots) have finished processing. - **`extra` string:** Arbitrary metadata string set via `test[extra]`. - **`type` string:** Driver type (WEBDRIVER, APPIUM). - **`steps` string:** Rendered HTML of the recorded step list. Omit with `skip_fields=steps`. - **`running` integer:** Completion percentage. Only present while a Codeless test is still running. GET `/v1/tests/:id` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/tests/:id" -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var test = await client.Tests.GetAsync(testId); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_test(test_id) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.tests.get_test(test_id) ``` ```php $api = new TestingBot\TestingBotAPI($key, $secret); $api->getJob($test_id); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingbotTest test = restApi.getTest(test_id); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const test = await api.getTestDetails(testId); ``` Response ```json { "id": 3, "name": "MyTest::testTitle", "session_id": "f7903f9e93e74fe1b0e924bf9d2ce9fc", "state": "COMPLETE", "success": false, "status_id": 0, "unknown": false, "status_message": "Failed asserting that 1 equals 0.", "created_at": "2011-07-30T23:21:23Z", "completed_at": "2011-07-30T23:22:23Z", "duration": 60, "browser": "iexplore", "browser_version": "8", "browser_long_version": "8.0.7601", "browser_display_name": "IE8", "os": "WINDOWS", "device_name": null, "platform_name": null, "build": null, "groups": ["testingbot.com"], "video": "https://s3.amazonaws.com/rectestingbot/sample.mp4", "thumbs": ["https://s3.amazonaws.com/euthumbtestingbot/3_f93782fji.jpg"], "logs": { "selenium": "https://s3-eu-west-1.amazonaws.com/eulogtestingbot/session.txt" }, "assets_available": true, "type": "WEBDRIVER" } ``` PUT `/v1/tests/:id` ## Update a test Updates a test's metadata after it's been recorded. Use this to mark a test as passed/failed from the test runner, attach groups/tags, set a build identifier, or add a status message. Accepts either a numeric test ID or a WebDriver session\_id. ### Arguments - **`id` string required:** Numeric test ID or WebDriver session\_id (UUID). - **`test[name]` string:** Human-readable test name. - **`test[success]` boolean:** true=pass, false=fail. - **`test[status_message]` string:** Failure reason or arbitrary status text. - **`test[extra]` string:** Arbitrary metadata. - **`test[build]` string:** Build identifier to associate this test with. - **`test[public]` boolean:** When true, makes the test publicly viewable via share URL. - **`groups` string:** Comma-separated string or array of tag/group names to attach to the test. - **`build` string:** Build identifier (alternative location to `test[build]`). ### Response fields - **`success` boolean:** Whether the operation succeeded. - **`errors` object:** Validation errors keyed by field name. Only present when `success` is false. - **`error` string:** Single human-readable reason, used by the older endpoints in place of `errors`. Only present when `success` is false. PUT `/v1/tests/:id` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/tests/:id" \ -X PUT \ -d "test[success]=1" \ -d "groups[]=regression" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); await client.Tests.UpdateAsync(testId, new TestUpdate { Success = true, Groups = new[] { "regression" } }); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.update_test(test_id, { name: 'new_name', success: true }) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.tests.update_test(test_id, status_message='..', passed=1, build='..', name='..') ``` ```php $api = new TestingBot\TestingBotAPI($key, $secret); $api->updateJob($test_id, ['name' => 'mytest', 'success' => true]); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); boolean success = restApi.updateTest(testId, details); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); await api.updateTest({ 'test[success]': '1', 'test[status_message]': 'failure reason' }, testId); ``` Response ```json { "success": true } ``` DELETE `/v1/tests/:id` ## Delete a test Permanently deletes a test session and every associated asset (video, logs, screenshots). This action cannot be undone. ### Arguments - **`id` string required:** Numeric test ID or WebDriver session\_id of the test to delete. DELETE `/v1/tests/:id` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/tests/:id" \ -X DELETE \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); await client.Tests.DeleteAsync(testId); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.delete_test(test_id) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.tests.delete_test(test_id) ``` ```php $api = new TestingBot\TestingBotAPI($key, $secret); $api->deleteJob($test_id); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); boolean success = restApi.deleteTest(testId); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); await api.deleteTest(testId); ``` Response ```json { "success": true } ``` GET `/v1/tests/:id/assets` ## List a test's assets Returns presigned download URLs for everything the session produced: video, per-type log files, HAR network capture, performance metrics and screenshots. URLs expire (video/logs/HAR after 1 hour, screenshots after 7 days) -- fetch this again for fresh ones. Ideal for archiving artifacts from CI after a run. ### Arguments - **`id` string required:** Numeric test ID or WebDriver session\_id. ### Response fields - **`session_id` string:** WebDriver session id of the test the assets belong to. - **`video` string:** Presigned video URL (valid 1 hour). Null when no video was recorded or assets were deleted. - **`logs` object:** Presigned log-file URLs keyed by log type (selenium, appium, logcat, browser, ...), each valid 1 hour. Empty object when none remain. - **`har` string:** Presigned HAR network-capture URL (valid 1 hour). Null unless network capture was enabled for the session. - **`performance` string:** Presigned performance-metrics log URL (valid 1 hour). Null unless performance data was collected. - **`screenshots` array of string:** Presigned screenshot URLs in capture order, each valid 7 days. GET `/v1/tests/:id/assets` [cURL](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/tests/:id/assets" -u key:secret ``` Response ```json { "session_id": "2b5a38f3c71e97a2265b52b9382db1c5", "video": "https://eurectestingbot.../screencast.mp4?X-Amz-Expires=3600...", "logs": { "selenium": "https://eulogtestingbot.../selenium.log?X-Amz-Expires=3600..." }, "har": null, "performance": null, "screenshots": [ "https://euthumbtestingbot.../2b5a38f3_1.jpg?X-Amz-Expires=604800..." ] } ``` DELETE `/v1/tests/:id/assets` ## Delete a test's assets Permanently deletes the session's stored files -- video, log files, HAR capture and screenshots -- ahead of the normal retention window, e.g. for compliance. The test result itself (status, metadata, command history) is kept; use DELETE /v1/tests/:id to remove the whole test. Idempotent: deleting a test with no remaining assets succeeds. ### Arguments - **`id` string required:** Numeric test ID or WebDriver session\_id. DELETE `/v1/tests/:id/assets` [cURL](https://testingbot.com#) Request ```bash $ curl -X DELETE "https://api.testingbot.com/v1/tests/:id/assets" -u key:secret ``` Response ```json { "success": true } ``` PUT `/v1/tests/:id/stop` ## Stop a running test Terminates an in-flight test session. The test is marked complete; any assets gathered (video, logs, screenshots) become available for download. Returns 404 if the test has already finished. ### Arguments - **`id` string required:** Numeric test ID or WebDriver session\_id of the test to stop. ### Response fields - **`success` boolean:** Whether the operation succeeded. - **`errors` object:** Validation errors keyed by field name. Only present when `success` is false. - **`error` string:** Single human-readable reason, used by the older endpoints in place of `errors`. Only present when `success` is false. PUT `/v1/tests/:id/stop` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/tests/:id/stop" \ -X PUT \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); await client.Tests.StopAsync(testId); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.stop_test(test_id) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.tests.stop_test(test_id) ``` ```php $api = new TestingBot\TestingBotAPI($key, $secret); $api->stopJob($test_id); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); boolean success = restApi.stopTest(testId); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); await api.stopTest(testId); ``` Response ```json { "success": true } ``` DELETE `/v1/tests` ## Bulk delete tests Deletes up to 25 tests -- and every associated asset (video, logs, screenshots) -- in one request. Each entry may be a numeric test ID or a WebDriver session\_id. Tests that cannot be found (or belong to another account) are reported in `missing`; the rest are gone for good, exactly like DELETE /v1/tests/:id. ### Arguments - **`ids` array required:** Up to 25 numeric test IDs or WebDriver session\_ids. DELETE `/v1/tests` [cURL](https://testingbot.com#) Request ```bash $ curl -X DELETE "https://api.testingbot.com/v1/tests?ids[]=123&ids[]=2b5a38f3c71e97a2265b52b9382db1c5" -u key:secret ``` Response ```json { "success": true, "deleted": ["123", "2b5a38f3c71e97a2265b52b9382db1c5"], "missing": [] } ``` POST `/v1/tests` ## Create a test record (manual sessions) Creates a TestCase record from outside the WebDriver/Appium flow. Mainly used by clients that want to log a manual or external test result against their TestingBot account. Most users will not need this — the normal flow is to start a Selenium/Appium session, and the record is created automatically. ### Arguments - **`test[name]` string required:** Test name. - **`test[test_environment_id]` integer required:** Browser environment the result is attributed to (`browser_id` from GET /v1/browsers). - **`test[success]` boolean:** true=pass, false=fail. - **`test[status_message]` string:** Failure reason or status string. - **`test[extra]` string:** Arbitrary metadata. - **`test[build]` string:** Build identifier. POST `/v1/tests` [cURL](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/tests" \ -u key:secret \ -d "test[name]=MyTest::testTitle" \ -d "test[test_environment_id]=1" \ -d "test[success]=true" ``` ```php $client = new TestingBot\Client($key, $secret); $client->tests()->create([ 'name' => 'MyTest::testTitle', 'test_environment_id' => 1, 'success' => true, ]); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); Map test = Map.of( "name", "MyTest::testTitle", "test_environment_id", 1, "success", true); boolean success = restApi.createTest(test); ``` POST `/v1/tests/{id}` ## Update a test (POST alias) Alias of `PUT /v1/tests/:id` for clients that cannot send PUT requests (some older HTTP libraries). Same body schema and behavior — see PUT for the canonical form. ### Arguments - **`id` string required:** Numeric test ID or WebDriver session\_id (UUID). - **`test[name]` string:** Human-readable test name. - **`test[success]` boolean:** true=pass, false=fail. - **`test[status_message]` string:** Failure reason or arbitrary status text. - **`test[extra]` string:** Arbitrary metadata. - **`test[build]` string:** Build identifier to associate this test with. - **`test[public]` boolean:** When true, makes the test publicly viewable via share URL. - **`groups` string:** Comma-separated string or array of tag/group names to attach to the test. POST `/v1/tests/{id}` [cURL](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/tests/{id}" \ -u key:secret ``` GET `/v1/builds` ## List your builds Returns a paginated list of test builds. A build is an aggregation of test cases that share the same `capabilities.build` identifier, useful for grouping CI runs. ### Arguments - **`offset` integer:** Skip this many builds from the start of the result set. - **`count` integer max=`500`:** Number of builds to return . - **`tag` string:** Only return builds carrying this tag (set tags with PUT /v1/builds/:id). ### Response fields - **`data` array of build objects:** Builds for this page. - **`meta` meta object:** — GET `/v1/builds` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/builds?offset=0&count=10" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var builds = await client.Builds.ListAsync(); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_builds(0, 10) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.build.get_builds(offset=0, limit=10) ``` ```php $api = new TestingBot\TestingBotAPI($key, $secret); $api->getBuilds(0, 10); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingbotBuildCollection builds = restApi.getBuilds(0, 10); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const builds = await api.getBuilds(offset, limit); ``` Response ```json { "data": [ { "id": 4331, "build_identifier": "first-build", "created_at": "2016-08-01T11:09:02.000Z", "updated_at": "2016-08-01T11:09:02.000Z" } ], "meta": { "offset": 0, "count": 10, "total": 3 } } ``` GET `/v1/builds/:id` ## Get tests for a build Returns all test cases that belong to a single build, with pagination. The build can be referenced by either its numeric internal ID or the string identifier you set via `capabilities.build`. ### Arguments - **`id` string required:** Numeric build ID or string build identifier. - **`offset` integer:** Skip this many tests in the build. - **`count` integer max=`500`:** Number of tests to return . - **`skip_fields` string:** Comma-separated fields to omit from each test (logs, thumbs). ### Response fields - **`data` array of test case objects:** Test sessions for this page. - **`meta` meta object:** — - **`realtime` object:** ActionCable channel details for live updates. Only present on the `since`-polling form of this endpoint. GET `/v1/builds/:id` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/builds/:id" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var tests = await client.Builds.GetTestsAsync(buildId); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_build(build_identifier) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.build.get_tests_for_build(build_id) ``` ```php $api = new TestingBot\TestingBotAPI($key, $secret); $api->getBuild($build_id); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingbotTestBuildCollection tests = restApi.getTestsForBuild(buildId); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const tests = await api.getTestsForBuild(buildId); ``` Response ```json { "data": [ { "id": 3, "name": "MyTest::testTitle", "session_id": "f7903f9e93e74fe1b0e924bf9d2ce9fc", "state": "COMPLETE", "success": false, "status_id": 0, "created_at": "2011-07-30T23:21:23Z", "completed_at": "2011-07-30T23:22:44Z", "duration": 13, "browser": "iexplore", "browser_version": "8", "browser_long_version": "8.0.7601", "browser_display_name": "IE8", "os": "WINDOWS", "build": "buildid", "video": "https://s3.amazonaws.com/rectestingbot/sample.mp4" } ], "meta": { "offset": 0, "count": 10, "total": 1 } } ``` DELETE `/v1/builds/:id` ## Delete a build Permanently deletes a build and every test, asset (video, logs, screenshots) attached to it. This action cannot be undone. ### Arguments - **`id` string required:** Numeric build ID or string build identifier to delete. DELETE `/v1/builds/:id` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/builds/:id" \ -X DELETE \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); await client.Builds.DeleteAsync(buildId); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.delete_build(build_identifier) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.build.delete_build(build_id) ``` ```php $api = new TestingBot\TestingBotAPI($key, $secret); $api->deleteBuild($build_id); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); boolean success = restApi.deleteBuild(buildId); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); await api.deleteBuild(buildId); ``` Response ```json { "success": true } ``` PUT `/v1/builds/:id` ## Update a build (rename, tags) Renames a build and/or replaces its tag set. Tags are short labels (letters, digits, dot, dash, underscore, colon, space; max 50 chars each, 10 per build) you can filter the builds list by (`GET /v1/builds?tag=...`) -- handy for marking CI runs as smoke, nightly, release-x and so on. Passing an empty `tags` array (JSON body) clears them. ### Arguments - **`id` string required:** Numeric build ID or string build identifier. - **`name` string:** New build identifier (capabilities.build value). - **`tags` array:** Replacement tag set; empty array clears all tags. ### Response fields - **`success` boolean:** Whether the build was updated. - **`build` object:** The updated build: id, build\_identifier and tags. PUT `/v1/builds/:id` [cURL](https://testingbot.com#) Request ```bash $ curl -X PUT "https://api.testingbot.com/v1/builds/:id" \ -d "name=nightly-2026-08-11" -d "tags[]=smoke" -d "tags[]=release-1.4" \ -u key:secret ``` Response ```json { "success": true, "build": { "id": 4711, "build_identifier": "nightly-2026-08-11", "tags": ["smoke", "release-1.4"] } } ``` DELETE `/v1/builds` ## Bulk delete builds Deletes up to 25 builds in one request. Each entry may be a numeric build ID or a string build identifier. Builds that cannot be found (or belong to another account) are reported in `missing`; the rest are deleted like DELETE /v1/builds/:id (tests keep their results, only the build grouping is removed). ### Arguments - **`ids` array required:** Up to 25 numeric build IDs or string build identifiers. DELETE `/v1/builds` [cURL](https://testingbot.com#) Request ```bash $ curl -X DELETE "https://api.testingbot.com/v1/builds?ids[]=4711&ids[]=nightly-2026-08-10" -u key:secret ``` Response ```json { "success": true, "deleted": ["4711", "nightly-2026-08-10"], "missing": [] } ``` POST `/v1/builds/{id}/ci` ## Link a build to a commit for CI status checks Associates a CI build with the commit/PR it ran for on a code host and posts an in-progress status check. Currently supports `github` (requires the TestingBot GitHub App installed on the repository owner). Call this at the start of a workflow run. ### Arguments - **`id` string required:** Build identifier (capabilities.build) for this run. - **`repo` string required:** Repository in owner/name form. - **`commit_sha` string required:** Head commit SHA the build ran for. For pull\_request events use github.event.pull\_request.head.sha (not the merge commit) so the check lands on the SHA branch protection evaluates. - **`provider` string:** Code host provider. Currently only "github". - **`branch` string:** Git branch name. - **`pull_request_number` integer:** Pull request number, if any. POST `/v1/builds/{id}/ci` [cURL](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/builds/{id}/ci" \ -u key:secret \ -d "repo=my-org/my-repo" \ -d "commit_sha=9fceb02d0ae598e95dc970b74767f19372d61af8" \ -d "branch=main" ``` PUT `/v1/builds/{id}/ci` ## Report a CI status-check build as finished Signals that the CI run for this build has finished, so the status check is updated with the aggregate pass/fail result. Call this after your tests complete. ### Arguments - **`id` string required:** Build identifier (capabilities.build) for this run. - **`provider` string:** Code host provider. Currently only "github". ### Response fields - **`success` boolean:** Whether the operation succeeded. - **`errors` object:** Validation errors keyed by field name. Only present when `success` is false. - **`error` string:** Single human-readable reason, used by the older endpoints in place of `errors`. Only present when `success` is false. PUT `/v1/builds/{id}/ci` [cURL](https://testingbot.com#) Request ```bash $ curl -X PUT "https://api.testingbot.com/v1/builds/{id}/ci" \ -u key:secret ``` [Previous User & Team Management](https://testingbot.com/support/api/user) [Next Insights](https://testingbot.com/support/api/insights) --- URL: https://testingbot.com/support/api/insights # Insights The data behind the Test Analytics dashboard, exposed so you can build your own reporting. - **Endpoint:** api.testingbot.com - **Version:** v1 - **Format:** JSON - **Auth:** [HTTP Basic](https://testingbot.com/support/api#authentication) ## Insights API The Insights API exposes read-only analytics over your test results so you can build your own dashboards. It returns the same data that powers the [Test Analytics](https://testingbot.com/members/tests) page in your dashboard: daily pass/fail trends, a headline summary, top failure messages, per-test flakiness, and the filter values to drive it all. For a raw list of individual test executions, use [GET /v1/tests](https://testingbot.com/support/api/tests#tests) instead. Every endpoint accepts a period via `start` and `end` (ISO-8601 or UNIX timestamps), or the `time_range` shorthand (`7d`, `30d`), defaulting to the last 30 days. Filter with `status`, `browser`, `build`, `group_ids` and `team` (comma-separated IDs discoverable via [GET /v1/insights/filters](https://testingbot.com#insights-filters)). Dates in results follow the server timezone (UTC). GET `/v1/insights/trends` ## Test trends Time-bucketed success / failure / unknown counts for your tests, the same series that powers the Test Analytics chart in the dashboard. Use `group_by` to aggregate by day (default), by browser (coverage), or as a single status roll-up. Filter by date range, status, browser, build, group and team. ### Arguments - **`start` string:** Period start. ISO-8601 (2026-06-01 or 2026-06-01T00:00:00Z) or a UNIX timestamp. Defaults to 30 days before `end`. - **`end` string:** Period end. ISO-8601 or a UNIX timestamp. Defaults to now. - **`time_range` string:** Relative window as a shorthand for `start`: 24h, 7d, 4w (units h/d/w). Ignored when `start` is given. - **`status` string:** Filter by test status. One of all (default), success, failure, unknown. - **`browser` array:** Filter by test environment (browser/OS/device) IDs, comma-separated. Discover IDs via /v1/insights/filters. - **`build` array:** Filter by build IDs, comma-separated. - **`group_ids` array:** Filter by group ("My Groups") IDs, comma-separated. - **`team` array:** Filter by team member user IDs (who ran the test), comma-separated. - **`name` string:** Filter to an exact test name. - **`group_by` string:** Aggregation dimension: date (daily buckets, default), browser, or status. ### Response fields - **`data` array of insight trend bucket objects:** Series buckets, newest first. - **`meta` insight trend meta object:** — GET `/v1/insights/trends` [cURL](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/insights/trends?group_by=date&time_range=30d" \ -u key:secret ``` ```javascript const auth = Buffer.from(`${key}:${secret}`).toString('base64'); const res = await fetch( 'https://api.testingbot.com/v1/insights/trends?group_by=date&time_range=30d', { headers: { Authorization: `Basic ${auth}` } } ); const trends = await res.json(); ``` ```python import requests res = requests.get( 'https://api.testingbot.com/v1/insights/trends', params={'group_by': 'date', 'time_range': '30d'}, auth=(key, secret)) trends = res.json() ``` Response ```json { "data": [ { "date": "2026-06-30", "success": 8, "failure": 0, "unknown": 0, "total": 8 }, { "date": "2026-06-26", "success": 2, "failure": 6, "unknown": 0, "total": 8 } ], "meta": { "group_by": "date", "start": "2026-06-04T00:00:00Z", "end": "2026-07-04T10:37:00Z", "status": "all", "success": 300, "failure": 190, "unknown": 22, "total": 512, "pass_rate": 0.5859 } } ``` GET `/v1/insights/summary` ## Test summary Headline totals (success, failure, unknown) plus pass and failure rates for the selected period and filters. Ideal for a single-number dashboard tile. ### Arguments - **`start` string:** Period start. ISO-8601 (2026-06-01 or 2026-06-01T00:00:00Z) or a UNIX timestamp. Defaults to 30 days before `end`. - **`end` string:** Period end. ISO-8601 or a UNIX timestamp. Defaults to now. - **`time_range` string:** Relative window as a shorthand for `start`: 24h, 7d, 4w (units h/d/w). Ignored when `start` is given. - **`status` string:** Filter by test status. One of all (default), success, failure, unknown. - **`browser` array:** Filter by test environment (browser/OS/device) IDs, comma-separated. Discover IDs via /v1/insights/filters. - **`build` array:** Filter by build IDs, comma-separated. - **`group_ids` array:** Filter by group ("My Groups") IDs, comma-separated. - **`team` array:** Filter by team member user IDs (who ran the test), comma-separated. - **`name` string:** Filter to an exact test name. ### Response fields - **`start` timestamp:** Period start (ISO-8601, UTC). - **`end` timestamp:** Period end (ISO-8601, UTC). - **`status` string:** Status filter applied. - **`total` integer:** Total tests in the period. - **`success` integer:** Passed tests. - **`failure` integer:** Failed tests. - **`unknown` integer:** Unknown-status tests. - **`pass_rate` number:** success / total, 0..1. - **`failure_rate` number:** failure / total, 0..1. GET `/v1/insights/summary` [cURL](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/insights/summary?time_range=30d" \ -u key:secret ``` ```javascript const auth = Buffer.from(`${key}:${secret}`).toString('base64'); const res = await fetch( 'https://api.testingbot.com/v1/insights/summary?time_range=30d', { headers: { Authorization: `Basic ${auth}` } } ); const summary = await res.json(); ``` ```python import requests res = requests.get( 'https://api.testingbot.com/v1/insights/summary', params={'time_range': '30d'}, auth=(key, secret)) summary = res.json() ``` Response ```json { "start": "2026-06-04T00:00:00Z", "end": "2026-07-04T10:37:00Z", "status": "all", "total": 512, "success": 300, "failure": 190, "unknown": 22, "pass_rate": 0.5859, "failure_rate": 0.3711 } ``` GET `/v1/insights/errors` ## Top failure messages Groups failed tests by their status message and returns the most common failures with occurrence counts. Always operates on failed tests, so the `status` filter is ignored. Paginated with offset/count. ### Arguments - **`start` string:** Period start. ISO-8601 (2026-06-01 or 2026-06-01T00:00:00Z) or a UNIX timestamp. Defaults to 30 days before `end`. - **`end` string:** Period end. ISO-8601 or a UNIX timestamp. Defaults to now. - **`time_range` string:** Relative window as a shorthand for `start`: 24h, 7d, 4w (units h/d/w). Ignored when `start` is given. - **`status` string:** Filter by test status. One of all (default), success, failure, unknown. - **`browser` array:** Filter by test environment (browser/OS/device) IDs, comma-separated. Discover IDs via /v1/insights/filters. - **`build` array:** Filter by build IDs, comma-separated. - **`group_ids` array:** Filter by group ("My Groups") IDs, comma-separated. - **`team` array:** Filter by team member user IDs (who ran the test), comma-separated. - **`name` string:** Filter to an exact test name. - **`offset` integer:** Skip this many error groups. - **`count` integer max=`500`:** Number of error groups to return . ### Response fields - **`data` array of insight error objects:** Error groups, most frequent first. - **`meta` insight error meta object:** — GET `/v1/insights/errors` [cURL](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/insights/errors?time_range=7d&count=10" \ -u key:secret ``` ```javascript const auth = Buffer.from(`${key}:${secret}`).toString('base64'); const res = await fetch( 'https://api.testingbot.com/v1/insights/errors?time_range=7d&count=10', { headers: { Authorization: `Basic ${auth}` } } ); const errors = await res.json(); ``` ```python import requests res = requests.get( 'https://api.testingbot.com/v1/insights/errors', params={'time_range': '7d', 'count': 10}, auth=(key, secret)) errors = res.json() ``` Response ```json { "data": [ { "error": "Element not found: #login", "count": 42 }, { "error": "Timeout after 30s waiting for element", "count": 17 } ], "meta": { "start": "2026-06-27T10:37:00Z", "end": "2026-07-04T10:37:00Z", "total_failures": 190, "distinct_errors": 12, "offset": 0, "count": 10 } } ``` GET `/v1/insights/test-cases` ## Test cases Groups your test runs by test name and returns per-test statistics: total runs, success/failure/unknown counts, pass rate, a flaky flag (both passes and failures seen), and average duration in seconds. Sort by total (default), pass\_rate, failure, flaky or avg\_duration. Paginated with offset/count. ### Arguments - **`start` string:** Period start. ISO-8601 (2026-06-01 or 2026-06-01T00:00:00Z) or a UNIX timestamp. Defaults to 30 days before `end`. - **`end` string:** Period end. ISO-8601 or a UNIX timestamp. Defaults to now. - **`time_range` string:** Relative window as a shorthand for `start`: 24h, 7d, 4w (units h/d/w). Ignored when `start` is given. - **`status` string:** Filter by test status. One of all (default), success, failure, unknown. - **`browser` array:** Filter by test environment (browser/OS/device) IDs, comma-separated. Discover IDs via /v1/insights/filters. - **`build` array:** Filter by build IDs, comma-separated. - **`group_ids` array:** Filter by group ("My Groups") IDs, comma-separated. - **`team` array:** Filter by team member user IDs (who ran the test), comma-separated. - **`name` string:** Filter to an exact test name. - **`sort` string:** Sort field. - **`sort_order` string:** Sort direction. - **`offset` integer:** Skip this many test cases. - **`count` integer max=`500`:** Number of test cases to return . ### Response fields - **`data` array of insight test case objects:** Per-test-name statistics. - **`meta` insight test case meta object:** — GET `/v1/insights/test-cases` [cURL](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/insights/test-cases?sort=flaky&count=20" \ -u key:secret ``` ```javascript const auth = Buffer.from(`${key}:${secret}`).toString('base64'); const res = await fetch( 'https://api.testingbot.com/v1/insights/test-cases?sort=flaky&count=20', { headers: { Authorization: `Basic ${auth}` } } ); const testCases = await res.json(); ``` ```python import requests res = requests.get( 'https://api.testingbot.com/v1/insights/test-cases', params={'sort': 'flaky', 'count': 20}, auth=(key, secret)) test_cases = res.json() ``` Response ```json { "data": [ { "name": "Login::testValidLogin", "total": 120, "success": 110, "failure": 8, "unknown": 2, "pass_rate": 0.9167, "flaky": true, "avg_duration": 34.2 } ], "meta": { "start": "2026-06-04T00:00:00Z", "end": "2026-07-04T10:37:00Z", "total": 87, "offset": 0, "count": 50, "sort": "flaky", "sort_order": "desc" } } ``` GET `/v1/insights/filters` ## Available filter values Returns the browsers, builds, groups and team members available on your account, so a dashboard can populate its filter dropdowns. The returned IDs are the values to pass to the other Insights endpoints (browser, build, group\_ids, team). ### Response fields - **`browsers` array of insight filter option objects:** Test environments used recently (pass ids as `browser`). - **`builds` array of object:** Builds ({id, identifier}); pass ids as `build`. - **`groups` array of insight filter option objects:** User-defined groups (pass ids as `group_ids`). - **`team_members` array of insight filter option objects:** Account owner + team members (pass ids as `team`). - **`statuses` array of string:** Valid status filter values. GET `/v1/insights/filters` [cURL](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/insights/filters" \ -u key:secret ``` ```javascript const auth = Buffer.from(`${key}:${secret}`).toString('base64'); const res = await fetch( 'https://api.testingbot.com/v1/insights/filters', { headers: { Authorization: `Basic ${auth}` } } ); const filters = await res.json(); ``` ```python import requests res = requests.get( 'https://api.testingbot.com/v1/insights/filters', auth=(key, secret)) filters = res.json() ``` Response ```json { "browsers": [ { "id": 1234, "name": "Chrome 126, Windows 11" }, { "id": 1288, "name": "Safari 17, macOS Sonoma" } ], "builds": [{ "id": 4331, "identifier": "ci-run-42" }], "groups": [{ "id": 7, "name": "smoke" }], "team_members": [{ "id": 9, "name": "Jane Doe" }], "statuses": ["success", "failure", "unknown"] } ``` [Previous Tests & Builds](https://testingbot.com/support/api/tests) [Next Screenshots](https://testingbot.com/support/api/screenshots) --- URL: https://testingbot.com/support/api/screenshots # Screenshots Kick off a screenshot run across browsers and fetch the images when it finishes. - **Endpoint:** api.testingbot.com - **Version:** v1 - **Format:** JSON - **Auth:** [HTTP Basic](https://testingbot.com/support/api#authentication) POST `/v1/screenshots` ## Capture a new screenshot batch Queues a cross-browser screenshot of a given URL at a specified resolution. Rendering is asynchronous: this returns the batch ID only, so poll GET /v1/screenshots/:id until its `state` is `done` to collect the images. ### Arguments - **`url` string required:** Page URL to screenshot. - **`resolution` string required:** Browser viewport (e.g. "1920x1080"). - **`browsers` array required:** Array of `{ browser, version, os }` triples (or `browser_id`s) to render with. - **`wait_time` integer:** Seconds to wait after page load before snapping (0-60). - **`fullpage` boolean:** Capture the entire scrollable page instead of just the viewport. - **`callback_url` string:** POST callback URL invoked when the batch finishes processing. POST `/v1/screenshots` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl -H 'Content-Type: application/json' -X POST "https://api.testingbot.com/v1/screenshots" \ -u key:secret \ -d '{ "url": "https://www.google.com", "resolution": "1280x1024", "browsers": [ { "browserName": "chrome", "version": 134, "os": "WIN10" }, { "browserName": "safari", "version": "17.2", "platformName": "iOS", "deviceName": "iPhone 15" } ] }' ``` ```csharp var client = new TestingBotClient(key, secret); var batch = await client.Screenshots.CaptureAsync(new ScreenshotRequest { Url = "https://www.google.com", Resolution = "1280x1024", BrowserIds = new[] { 1, 22 } }); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.take_screenshots({ url: 'https://www.google.com', resolution: '1280x1024', browsers: [{ browserName: 'chrome', version: 134, os: 'WIN10' }] }) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.screenshots.take_screenshots('https://www.google.com', '1280x1024', [1, 22]) ``` ```php $client = new TestingBot\Client($key, $secret); $batch = $client->screenshots()->create('https://example.com', [1, 2], [ 'resolution' => '1920x1080', ]); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); Map params = new HashMap<>(); params.put("url", "https://www.google.com"); params.put("resolution", "1280x1024"); TestingbotScreenshot screenshots = restApi.createScreenshots(params); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const screenshots = await api.takeScreenshot( 'https://example.com', [{ browserName: 'chrome', version: 'latest', os: 'WIN11' }], '1920x1080' ); ``` Response ```json { "id": 3454, "wait_time": 0, "resolution": "1280x1024", "url": "https://www.google.com" } ``` GET `/v1/screenshots/{id}` ## Get screenshot batch detail Returns per-browser screenshot results for a single batch, including thumbnail/image URLs and processing state. ### Arguments - **`id` integer required:** Numeric screenshot batch ID. - **`excludeIds` string:** Comma-separated screenshot IDs to exclude (useful for delta-fetch). ### Response fields - **`id` integer:** Unique screenshot batch ID. - **`url` string:** URL the screenshots were taken from. - **`resolution` string:** Browser viewport resolution. - **`state` string:** Batch state: `processing` until every browser has finished, then `done`. - **`created_at` timestamp:** — - **`screenshots` array of screenshot result objects:** Per-browser results with download URLs. GET `/v1/screenshots/{id}` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/screenshots/{id}" -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var batch = await client.Screenshots.GetAsync(batchId); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_screenshots(screenshot_id) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.screenshots.get_screenshot(screenshot_id) ``` ```php $client = new TestingBot\Client($key, $secret); $batch = $client->screenshots()->get($screenshotId); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingbotScreenshot screenshot = restApi.getScreenshot(screenshotId); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const screenshotResult = await api.retrieveScreenshots(screenshotJobId); ``` Response ```json { "id": 3454, "url": "https://www.google.com", "resolution": "1280x1024", "created_at": "2017-10-13T14:47:41.000Z", "state": "done", "screenshots": [ { "image_url": "https://....", "thumb_url": "https://....", "state": "done", "id": "9c7deea4-6f23-4041-842b-39cccee447fa", "session_id": 0, "created_at": "2017-10-13T14:47:44.000Z", "os": "SEQUOIA", "browser_name": "googlechrome", "browser_version": "138", "browser_id": 1146, "device_name": null, "platform_name": null } ] } ``` GET `/v1/screenshots` ## List screenshot batches Returns batches of cross-browser screenshots the account has queued historically. ### Arguments - **`offset` integer:** Skip this many batches. - **`count` integer:** Number of batches to return. ### Response fields - **`data` array of screenshot summary objects:** Screenshot batches for this page. - **`meta` meta object:** — GET `/v1/screenshots` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/screenshots" -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var history = await client.Screenshots.ListAsync(); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_screenshots_history(0, 10) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.screenshots.get_screenshots(offset=0, limit=10) ``` ```php $client = new TestingBot\Client($key, $secret); $batches = $client->screenshots()->list(0, 10); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingbotScreenshotCollection screenshots = restApi.getScreenshots(); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const screenshots = await api.getScreenshotList(); ``` Response ```json { "data": [ { "id": 3454, "url": "https://google.com", "resolution": "1280x1024", "created_at": "2017-10-13T14:47:41.000Z" }, { "id": 3452, "url": "https://testingbot.com", "resolution": "1280x1024", "created_at": "2017-10-12T09:22:03.000Z" } ], "meta": { "offset": 0, "count": 3, "total": 3 } } ``` [Previous Insights](https://testingbot.com/support/api/insights) [Next Tunnel](https://testingbot.com/support/api/tunnel) --- URL: https://testingbot.com/support/api/tunnel # Tunnel Manage the secure tunnels that let the grid reach your staging and internal environments. - **Endpoint:** api.testingbot.com - **Version:** v1 - **Format:** JSON - **Auth:** [HTTP Basic](https://testingbot.com/support/api#authentication) GET `/v1/tunnel/list` ## List active tunnels Returns every TestingBot Tunnel currently running under the account. Use this to monitor running tunnels in CI dashboards or to find a tunnel ID before tearing it down. ### Response fields - **`id` integer:** Unique numeric tunnel ID. - **`identifier` string:** The `--tunnel-identifier` the tunnel was started with, or null when it was started without one. Use this to tell concurrent tunnels on the same account apart. - **`state` string:** Tunnel lifecycle state (READY, STOPPED, …). - **`private_ip` string:** Internal IP of the tunnel VM. - **`ip` string:** Public IP of the tunnel VM. - **`requested_at` timestamp:** When the tunnel was requested. - **`user_id` integer:** ID of the user the tunnel belongs to. GET `/v1/tunnel/list` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/tunnel/list" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var tunnels = await client.Tunnels.ListAsync(); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_tunnels ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.tunnel.get_tunnels() ``` ```php $api = new TestingBot\TestingBotAPI($key, $secret); $api->getTunnels(); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); ArrayList tunnels = restApi.getTunnels(); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const tunnels = await api.getTunnelList(); ``` Response ```json [ { "id": 1, "identifier": "my-ci-tunnel", "state": "READY", "ip": "xx", "private_ip": "xx", "requested_at": "2026-05-13 01:34:23" } ] ``` DELETE `/v1/tunnel/{id}` ## Stop a tunnel Tears down a running tunnel. Use this in CI teardown steps to release the slot for the next run; idle tunnels also self-terminate after the configured timeout. ### Arguments - **`id` integer required:** Numeric tunnel ID to stop. DELETE `/v1/tunnel/{id}` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/tunnel/{id}" \ -X DELETE \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); await client.Tunnels.StopAsync(tunnelId); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.delete_tunnel(tunnel_id) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.tunnel.delete_tunnel(tunnel_id) ``` ```php $api = new TestingBot\TestingBotAPI($key, $secret); $api->deleteTunnel($tunnelID); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); boolean success = restApi.deleteTunnel(tunnelId); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); await api.deleteTunnel(tunnelId); ``` Response ```json { "success": true } ``` GET `/v1/tunnel` ## Get the user's active tunnel Returns the first active tunnel for the authenticated account (regardless of ID). Convenience endpoint when you only run one tunnel at a time. ### Response fields - **`id` integer:** Unique numeric tunnel ID. - **`state` string:** Tunnel lifecycle state (READY, STOPPED, …). - **`private_ip` string:** Internal IP of the tunnel VM, used to chain the tunnel with BrowserMob. - **`ip` string:** Public IP of the tunnel VM. Only present once `state` is READY. - **`version` string:** Latest tunnel client version TestingBot supports. - **`ssh_fingerprint` string:** OpenSSH SHA-256 fingerprint (`SHA256:...`) of the host key the tunnel VM presents on `ip`. The tunnel client authenticates to that VM with the account key and secret as SSH username and password, so it requires this key before connecting rather than trusting whatever answers. Only present once `state` is READY; absent for tunnels whose key was not recorded, which clients treat as unverified. GET `/v1/tunnel` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/tunnel" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var tunnel = await client.Tunnels.GetActiveAsync(); ``` ```php $client = new TestingBot\Client($key, $secret); $tunnel = $client->tunnels()->getActive(); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingbotTunnel tunnel = restApi.getTunnel(); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const tunnel = await api.getTunnel(); ``` DELETE `/v1/tunnel` ## Stop the user's active tunnel Tears down whichever tunnel is currently active on the account, without needing to specify an ID. Useful when you only ever run one tunnel and want a simple cleanup call. DELETE `/v1/tunnel` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl -X DELETE "https://api.testingbot.com/v1/tunnel" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); await client.Tunnels.StopActiveAsync(); ``` ```php $client = new TestingBot\Client($key, $secret); $client->tunnels()->deleteActive(); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); boolean success = restApi.deleteTunnel(); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); await api.deleteActiveTunnel(); ``` POST `/v1/tunnel/create` ## Create a tunnel Launches a new TestingBot Tunnel VM and returns its connection metadata. Most users start tunnels via the tunnel client/CLI rather than this endpoint directly. POST `/v1/tunnel/create` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/tunnel/create" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var tunnel = await client.Tunnels.CreateAsync(); ``` ```php $client = new TestingBot\Client($key, $secret); $tunnel = $client->tunnels()->create(); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingbotTunnel tunnel = restApi.createTunnel(); ``` GET `/v1/tunnel/isalive-check` ## Liveness check Unauthenticated ping endpoint used by the tunnel binary itself to confirm it can reach the TestingBot API. Always returns `{ success: true }`. ### Response fields - **`success` boolean:** Whether the operation succeeded. - **`errors` object:** Validation errors keyed by field name. Only present when `success` is false. - **`error` string:** Single human-readable reason, used by the older endpoints in place of `errors`. Only present when `success` is false. GET `/v1/tunnel/isalive-check` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Java](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/tunnel/isalive-check" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var alive = await client.Tunnels.IsAliveAsync(); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); boolean alive = restApi.isTunnelAlive(); ``` GET `/v1/tunnel/{id}` ## Get a specific tunnel Returns the current state of a tunnel by ID. Returns 500 with the underlying error if the tunnel is in a transition state or no longer exists. ### Arguments - **`id` integer required:** Numeric tunnel ID returned by /v1/tunnel/list or /v1/tunnel/create. ### Response fields - **`id` integer:** Unique numeric tunnel ID. - **`state` string:** Tunnel lifecycle state (READY, STOPPED, …). - **`private_ip` string:** Internal IP of the tunnel VM, used to chain the tunnel with BrowserMob. - **`ip` string:** Public IP of the tunnel VM. Only present once `state` is READY. - **`version` string:** Latest tunnel client version TestingBot supports. - **`ssh_fingerprint` string:** OpenSSH SHA-256 fingerprint (`SHA256:...`) of the host key the tunnel VM presents on `ip`. The tunnel client authenticates to that VM with the account key and secret as SSH username and password, so it requires this key before connecting rather than trusting whatever answers. Only present once `state` is READY; absent for tunnels whose key was not recorded, which clients treat as unverified. GET `/v1/tunnel/{id}` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/tunnel/{id}" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var tunnel = await client.Tunnels.GetAsync(tunnelId); ``` ```php $client = new TestingBot\Client($key, $secret); $tunnel = $client->tunnels()->get($tunnelId); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingbotTunnel tunnel = restApi.getTunnel(tunnelId); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const tunnel = await api.getTunnelById(tunnelId); ``` [Previous Screenshots](https://testingbot.com/support/api/screenshots) [Next Codeless Automation & TestLab](https://testingbot.com/support/api/codeless) --- URL: https://testingbot.com/support/api/codeless # Codeless Automation & TestLab Drive the codeless test builder and the TestLab suites that group those tests, end to end. - **Endpoint:** api.testingbot.com - **Version:** v1 - **Format:** JSON - **Auth:** [HTTP Basic](https://testingbot.com/support/api#authentication) GET `/v1/lab` ## List your Codeless tests Paginated list of every Codeless test (a.k.a. Lab test) on the account. Codeless tests are recorded in TestingBot's in-browser recorder and can be scheduled on a cron, fired via API, or chained into suites. ### Arguments - **`offset` integer:** Skip this many tests from the start of the result set. - **`count` integer max=`500`:** Number of tests to return . ### Response fields - **`data` array of lab test objects:** — - **`meta` meta object:** — GET `/v1/lab` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/lab" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var tests = await client.CodelessTests.ListAsync(); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_lab_tests(0, 10) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.lab.get_tests(offset=0, limit=10) ``` ```php $client = new TestingBot\Client($key, $secret); $tests = $client->lab()->listTests(0, 10); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingbotLabTestCollection labTests = restApi.getLabTests(); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const codelessTests = await api.getCodelessTests(); ``` Response ```json { "data": [ { "id": 215, "enabled": false, "alerts": [], "url": "https://testingbot.com/", "name": "testingbot", "created_at": "2012-03-12T20:03:45Z", "updated_at": "2012-03-13T20:26:43Z", "last_run": "2012-03-13T06:50:48Z", "browsers": [ { "name": "firefox", "version": "10", "os": "LINUX" } ] } ], "meta": { "offset": 0, "count": 10, "total": 25 } } ``` GET `/v1/lab/{id}` ## Get a specific Codeless test Returns a single Codeless test's configuration: schedule, alerts, attached browsers, and timestamps. Use the steps endpoint to retrieve the actual test recording. ### Arguments - **`id` integer required:** Numeric Codeless test ID. ### Response fields - **`id` integer:** Unique numeric Codeless test ID. - **`name` string:** Test name. - **`url` string:** Target URL the test runs against. - **`enabled` boolean:** Whether scheduled runs are active. - **`cron` string:** Cron expression for scheduled runs; null if unscheduled. - **`created_at` timestamp:** — - **`updated_at` timestamp:** — - **`last_run` timestamp:** Most recent run timestamp. - **`alerts` array of lab alert objects:** Configured alert destinations. - **`browsers` array of browser objects:** Browsers this test is configured to run on. GET `/v1/lab/{id}` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/lab/{id}" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var test = await client.CodelessTests.GetAsync(testId); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_lab_test(lab_test_id) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.lab.get_test(lab_test_id) ``` ```php $client = new TestingBot\Client($key, $secret); $test = $client->lab()->getTest($labTestId); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingbotLabTest labTest = restApi.getLabTest(labTestId); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const codelessTest = await api.getCodelessTest(labTestId); ``` Response ```json { "id": 215, "enabled": false, "alerts": [], "url": "https://testingbot.com/", "name": "testingbot", "created_at": "2012-03-12T20:03:45Z", "updated_at": "2012-03-13T20:26:43Z", "last_run": "2012-03-13T06:50:48Z", "browsers": [ { "name": "firefox", "version": "10", "os": "LINUX" } ] } ``` POST `/v1/lab` ## Create a Codeless test Creates a new Codeless test. Pass `test[name]` and either `test[url]` (target URL the test visits) or `file` (Selenium IDE export to import). Optional `test[ai_prompt]` lets you describe what the AI test agent should verify in plain English. ### Arguments - **`test[name]` string:** Test name. - **`test[url]` string:** Target URL to test (required if no `file` is uploaded). - **`test[cron]` string:** Cron expression for scheduled runs. - **`test[screenshot]` boolean:** Take screenshots at every step. - **`test[video]` boolean:** Record a video of the test. - **`test[idletimeout]` integer:** Seconds of idle time before the runner aborts the test. - **`test[screenresolution]` string:** Browser viewport (e.g. "1920x1080"). - **`test[ai_prompt]` string:** Plain-English instruction for the AI test agent. - **`file` file:** Selenium IDE `.side` export to import as the test's steps. POST `/v1/lab` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/lab" \ -u key:secret \ -d "test[name]=test" \ -d "test[cron]=31 * * * *" ``` ```csharp var client = new TestingBotClient(key, secret); long testId = await client.CodelessTests.CreateAsync(new CodelessTestCreate { Name = "My test", Url = "https://example.com" }); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.create_lab_test({ "test[name]" => 'test', "test[cron]" => '31 * * * *' }) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.lab.create_test(name='test', cron='31 * * * *') ``` ```php $client = new TestingBot\Client($key, $secret); $test = $client->lab()->createTest(['name' => 'smoke', 'url' => 'https://example.com']); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); Map fields = new HashMap<>(); fields.put("name", "test"); fields.put("cron", "31 * * * *"); TestingbotLabCreateAck ack = restApi.createLabTest(fields); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const result = await api.createCodelessTest({ name: 'My Codeless Test', url: 'https://example.com', ai_prompt: 'Test the login flow', cron: '0 0 * * *', screenshot: true, video: false, idletimeout: 60, screenresolution: '1920x1080' }); ``` Response ```json { "success": true, "lab_test_id": 59392 } ``` DELETE `/v1/lab/{id}` ## Delete a Codeless test Permanently deletes a Codeless test and its steps. Test runs in history remain. ### Arguments - **`id` integer required:** Numeric Codeless test ID to delete. DELETE `/v1/lab/{id}` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/lab/{id}" \ -X DELETE \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); await client.CodelessTests.DeleteAsync(testId); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.delete_lab_test(lab_test_id) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.lab.delete_test(lab_test_id) ``` ```php $client = new TestingBot\Client($key, $secret); $client->lab()->deleteTest($labTestId); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); boolean success = restApi.deleteLabTest(labTestId); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const result = await api.deleteCodelessTest(testId); ``` Response ```json { "success": true } ``` PUT `/v1/lab/{id}` ## Update a Codeless test Updates a Codeless test's metadata: name, target URL, cron schedule, enabled state. Accepts either a numeric Codeless test ID or a WebDriver session\_id of a test run that originated from this Codeless test. ### Arguments - **`id` string required:** Numeric Codeless test ID or WebDriver session\_id. - **`test[name]` string:** New test name. - **`test[url]` string:** New target URL. - **`test[cron]` string:** New cron expression. - **`test[enabled]` boolean:** Enable or pause scheduled runs. ### Response fields - **`success` boolean:** Whether the operation succeeded. - **`errors` object:** Validation errors keyed by field name. Only present when `success` is false. - **`error` string:** Single human-readable reason, used by the older endpoints in place of `errors`. Only present when `success` is false. PUT `/v1/lab/{id}` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/lab/{id}" \ -X PUT \ -d "test[cron]=* * * * *" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); await client.CodelessTests.UpdateAsync(testId, new CodelessTestUpdate { Name = "Renamed test" }); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.update_lab_test(lab_test_id, { "test[cron]" => '* * * * *' }) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.lab.update_test(lab_test_id, cron='* * * * *') ``` ```php $client = new TestingBot\Client($key, $secret); $client->lab()->updateTest($labTestId, ['name' => 'smoke test']); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); Map fields = new HashMap<>(); fields.put("cron", "* * * * *"); boolean success = restApi.updateLabTest(labTestId, fields); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const result = await api.updateCodelessTest({ test: { name: 'Updated Test Name', cron: '0 12 * * *' } }, testId); ``` Response ```json { "success": true } ``` POST `/v1/lab/{id}/schedule` ## Set or update a Codeless test schedule Schedules a Codeless test to run on a recurring interval. Choose between once / daily / weekly presets or a raw cron expression. ### Arguments - **`id` integer required:** Numeric Codeless test ID. - **`type` string:** Schedule preset; use "custom" with `cronFormat` for fine control. - **`day` string:** Date (YYYY-MM-DD) for "once" or weekday for "weekly". - **`hour` string:** Time (HH:MM) for "once", "daily", or "weekly". - **`cronFormat` string:** Raw cron expression (5-field) used when `type=custom`. POST `/v1/lab/{id}/schedule` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/lab/{id}/schedule" \ -u key:secret \ -d "type=daily" -d "hour=00:01" ``` ```csharp var client = new TestingBotClient(key, secret); await client.CodelessTests.ScheduleAsync(testId, new CodelessSchedule { Type = CodelessScheduleType.Daily, Hour = "09:00" }); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.schedule_lab_test(lab_test_id, { type: 'daily', hour: '00:01' }) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.lab.schedule(lab_test_id, 'daily', hour='00:01') ``` ```php $client = new TestingBot\Client($key, $secret); $client->lab()->schedule($labTestId, ['cron' => '0 * * * *']); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); Map params = new HashMap<>(); params.put("type", "daily"); params.put("hour", "00:01"); boolean success = restApi.scheduleLabTest(labTestId, params); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const result = await api.scheduleCodelessTest(labTestId, { type: 'daily', hour: '00:01' }); ``` Response ```json { "success": true } ``` POST `/v1/lab/{id}/alert` ## Add an alert to a Codeless test Adds a notification channel (email, SMS, or callback URL) that fires when a scheduled run fails. Use PUT to modify an existing alert. ### Arguments - **`id` integer required:** Numeric Codeless test ID. - **`kind` string required:** Alert channel. - **`level` string required:** When to send (every failure vs. daily digest). - **`content` string required:** Destination — email address, callback URL, or phone number. POST `/v1/lab/{id}/alert` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/lab/{id}/alert" \ -u key:secret \ -d "kind=EMAIL" -d "level=IMMEDIATELY" -d "content=alerts@example.com" ``` ```csharp var client = new TestingBotClient(key, secret); await client.CodelessTests.AddAlertAsync(testId, new CodelessAlertInput { Kind = AlertKind.Email, Level = AlertLevel.Immediately, Content = "dev@example.com" }); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.add_lab_test_alert(lab_test_id, { kind: 'EMAIL', level: 'IMMEDIATELY', content: 'alerts@example.com' }) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.lab.add_alert(lab_test_id, 'EMAIL', 'IMMEDIATELY', 'alerts@example.com') ``` ```php $client = new TestingBot\Client($key, $secret); $client->lab()->addAlert($labTestId, ['content' => 'ops@example.com']); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); Map params = new HashMap<>(); params.put("kind", "EMAIL"); params.put("level", "IMMEDIATELY"); params.put("content", "alerts@example.com"); boolean success = restApi.addLabTestAlert(labTestId, params); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const result = await api.createCodelessAlert(labTestId, { kind: 'EMAIL', level: 'IMMEDIATELY', content: 'alerts@example.com' }); ``` Response ```json { "success": true } ``` POST `/v1/lab/{id}/report` ## Add a daily report config to a Codeless test Configures a recurring email report summarising pass/fail rate for this Codeless test. Use PUT to update. ### Arguments - **`id` integer required:** Numeric Codeless test ID. - **`email` string required:** Email address that receives the report. - **`cron` string:** Cron expression for when to send the report (defaults to daily). POST `/v1/lab/{id}/report` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/lab/{id}/report" \ -u key:secret \ -d "email=reports@example.com" -d "cron=0 9 * * *" ``` ```csharp var client = new TestingBotClient(key, secret); await client.CodelessTests.AddReportAsync(testId, new CodelessReportInput { Email = "dev@example.com" }); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.create_lab_test_report(lab_test_id, { email: 'reports@example.com', cron: '0 9 * * *' }) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.lab.add_report(lab_test_id, 'reports@example.com', cron='0 9 * * *') ``` ```php $client = new TestingBot\Client($key, $secret); $client->lab()->addReport($labTestId, ['email' => 'ops@example.com']); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); Map params = new HashMap<>(); params.put("email", "reports@example.com"); params.put("cron", "0 9 * * *"); boolean success = restApi.createLabTestReport(labTestId, params); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const result = await api.createCodelessReport(labTestId, { email: 'reports@example.com', cron: '0 9 * * *' }); ``` Response ```json { "success": true } ``` POST `/v1/lab/{id}/steps` ## Replace the Codeless test's steps Deletes the test's existing steps and replaces them with the provided array. Useful for programmatic editing of recorded tests. ### Arguments - **`id` integer required:** Numeric Codeless test ID. - **`steps` array required:** Ordered list of steps; each is `{ order:, cmd:, locator:, value: }`. POST `/v1/lab/{id}/steps` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/lab/{id}/steps" \ -u key:secret \ -H 'Content-Type: application/json' \ -d '{ "steps": [{ "order": 0, "cmd": "open", "locator": "/", "value": "" }] }' ``` ```csharp var client = new TestingBotClient(key, secret); await client.CodelessTests.SetStepsAsync(testId, new[] { new CodelessStepInput { Order = 1, Command = "open", Value = "/" } }); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.set_lab_test_steps(lab_test_id, { steps: [{ order: 0, cmd: 'open', locator: '/', value: '' }] }) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.lab.set_steps(lab_test_id, [{'order': 0, 'cmd': 'open', 'locator': '/', 'value': ''}]) ``` ```php $client = new TestingBot\Client($key, $secret); $client->lab()->setSteps($labTestId, [ ['order' => 0, 'cmd' => 'open', 'locator' => '/', 'value' => ''], ]); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); List steps = Arrays.asList("open", "click"); boolean success = restApi.setLabTestSteps(labTestId, steps); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const result = await api.addCodelessStep(labTestId, { steps: [{ order: 0, cmd: 'open', locator: '/', value: '' }] }); ``` Response ```json { "success": true } ``` GET `/v1/lab/{id}/steps` ## List Codeless test steps Returns the recorded Selenium-IDE steps for a Codeless test, with pagination. ### Arguments - **`id` integer required:** Numeric Codeless test ID. - **`offset` integer:** Skip this many steps. - **`count` integer:** Number of steps to return. ### Response fields - **`data` array of lab test step objects:** — - **`meta` meta object:** — GET `/v1/lab/{id}/steps` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/lab/{id}/steps" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var steps = await client.CodelessTests.GetStepsAsync(testId); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_lab_test_steps(lab_test_id) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.lab.get_steps(lab_test_id) ``` ```php $client = new TestingBot\Client($key, $secret); $steps = $client->lab()->getSteps($labTestId, 0, 10); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingbotLabTestStepCollection steps = restApi.getLabTestSteps(labTestId); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const steps = await api.getCodelessSteps(labTestId); ``` Response ```json { "data": [ { "test_order": 0, "cmd": "open", "locator": "/", "value": "", "created_at": "2019-05-02T14:33:48.000Z", "updated_at": "2019-05-02T14:33:48.000Z" }, { "test_order": 1, "cmd": "type", "locator": "id=username", "value": "test", "created_at": "2019-05-02T14:33:48.000Z", "updated_at": "2019-05-02T14:33:48.000Z" }, { "test_order": 2, "cmd": "type", "locator": "id=password", "value": "test", "created_at": "2019-05-02T14:33:48.000Z", "updated_at": "2019-05-02T14:33:48.000Z" } ], "meta": { "offset": 0, "count": 3, "total": 3 } } ``` GET `/v1/lab/{id}/browsers` ## Get browsers for a Codeless test Returns the list of browsers this Codeless test is configured to run on. ### Arguments - **`id` integer required:** Numeric Codeless test ID. ### Response fields - **`name` string:** Browser identifier (e.g. "firefox", "chrome"). - **`version` string:** Browser version the test runs against. - **`os` string:** Operating system the test runs on (e.g. "WINDOWS", "MAC", "LINUX"). GET `/v1/lab/{id}/browsers` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/lab/{id}/browsers" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var browsers = await client.CodelessTests.GetBrowsersAsync(testId); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_lab_test_browsers(lab_test_id) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.lab.get_browsers(lab_test_id) ``` ```php $client = new TestingBot\Client($key, $secret); $browsers = $client->lab()->getBrowsers($labTestId); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); List browsers = restApi.getLabTestBrowsers(labTestId); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const browsers = await api.getCodelessBrowsers(labTestId); ``` Response ```json [ { "name": "firefox", "version": "41", "os": "VISTA" } ] ``` POST `/v1/lab/{id}/browsers` ## Update browsers for a Codeless test Replaces the entire browser set attached to this Codeless test. Pass `browser_ids` as a comma-separated list of IDs from `/v1/browsers`. ### Arguments - **`id` integer required:** Numeric Codeless test ID. - **`browser_ids` string required:** Comma-separated list of browser\_ids the test should run on. POST `/v1/lab/{id}/browsers` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/lab/{id}/browsers" \ -u key:secret \ -d "browser_ids=1,5,12" ``` ```csharp var client = new TestingBotClient(key, secret); await client.CodelessTests.SetBrowsersAsync(testId, new[] { 1, 22 }); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.set_lab_test_browsers(lab_test_id, { browser_ids: '1,5,12' }) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.lab.set_browsers(lab_test_id, '1,5,12') ``` ```php $client = new TestingBot\Client($key, $secret); $client->lab()->setBrowsers($labTestId, [1, 2]); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); boolean success = restApi.setLabTestBrowsers(labTestId, "1,5,12"); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const result = await api.setCodelessBrowsers(labTestId, { browser_ids: '1,5,12' }); ``` Response ```json { "success": true } ``` POST `/v1/lab/{id}/trigger` ## Run a specific Codeless test Triggers an immediate run of a Codeless test on the browsers configured for it. Returns a `job_id` you can poll with `GET /v1/jobs/:id`. ### Arguments - **`id` integer required:** Numeric Codeless test ID to run. - **`url` string:** Override the test's base URL for this run only. POST `/v1/lab/{id}/trigger` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/lab/{id}/trigger" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var result = await client.CodelessTests.TriggerAsync(testId); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.trigger_lab_test(lab_test_id) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.lab.trigger(lab_test_id) ``` ```php $client = new TestingBot\Client($key, $secret); $run = $client->lab()->trigger($labTestId); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingbotLabRunAck ack = restApi.triggerLabTest(labTestId); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const result = await api.triggerCodelessTest(labTestId); ``` Response ```json { "success": true, "job_id": 14 } ``` PUT `/v1/lab/{id}/stop` ## Stop a running Codeless test Force-stops an in-flight Codeless test run. Optional `browser_id` stops only the run on a specific browser. ### Arguments - **`id` integer required:** Numeric Codeless test ID. - **`browser_id` integer:** Only stop the run on this browser\_id (omit to stop all). ### Response fields - **`success` boolean:** Whether the operation succeeded. - **`errors` object:** Validation errors keyed by field name. Only present when `success` is false. - **`error` string:** Single human-readable reason, used by the older endpoints in place of `errors`. Only present when `success` is false. PUT `/v1/lab/{id}/stop` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl -X PUT "https://api.testingbot.com/v1/lab/{id}/stop" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); await client.CodelessTests.StopAsync(testId); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.stop_lab_test(lab_test_id) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.lab.stop(lab_test_id) ``` ```php $client = new TestingBot\Client($key, $secret); $client->lab()->stop($labTestId); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); boolean success = restApi.stopLabTest(labTestId); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const result = await api.stopCodelessTest(labTestId); ``` Response ```json { "success": true } ``` POST `/v1/lab/trigger_all` ## Run all Codeless tests Queues a run of every Codeless test on the account. Returns a job\_id that aggregates the results — poll via `GET /v1/jobs/:id`. Optional `url` overrides each test's base URL for this run. ### Arguments - **`url` string:** Override base URL for every queued test. POST `/v1/lab/trigger_all` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/lab/trigger_all" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var result = await client.CodelessTests.TriggerAllAsync(); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.trigger_all_lab_tests ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.lab.trigger_all() ``` ```php $client = new TestingBot\Client($key, $secret); $client->lab()->triggerAll(); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingbotLabRunAck ack = restApi.triggerAllLabTests(); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const result = await api.triggerAllCodelessTests(); ``` Response ```json { "success": true, "job_id": 14 } ``` PUT `/v1/lab/{id}/alert` ## Update an existing alert Replaces the alert configuration on a Codeless test. POST to /alert creates one; PUT updates it. ### Arguments - **`id` integer required:** Numeric Codeless test ID. - **`kind` string:** Alert channel. - **`level` string:** When to send. - **`content` string:** Updated destination value. ### Response fields - **`success` boolean:** Whether the operation succeeded. - **`errors` object:** Validation errors keyed by field name. Only present when `success` is false. - **`error` string:** Single human-readable reason, used by the older endpoints in place of `errors`. Only present when `success` is false. PUT `/v1/lab/{id}/alert` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl -X PUT "https://api.testingbot.com/v1/lab/{id}/alert" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); await client.CodelessTests.UpdateAlertAsync(testId, new CodelessAlertInput { Content = "ops@example.com" }); ``` ```php $client = new TestingBot\Client($key, $secret); $client->lab()->updateAlert($labTestId, ['content' => 'ops@example.com']); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); Map alert = Map.of("content", "ops@example.com"); boolean success = restApi.updateLabTestAlert(labTestId, alert); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); await api.updateCodelessAlert(testId, { content: 'ops@example.com' }); ``` PUT `/v1/lab/{id}/report` ## Update a daily report config Replaces the report destination/schedule for a Codeless test. ### Arguments - **`id` integer required:** Numeric Codeless test ID. - **`email` string:** Updated email address. - **`cron` string:** Updated cron expression. ### Response fields - **`success` boolean:** Whether the operation succeeded. - **`errors` object:** Validation errors keyed by field name. Only present when `success` is false. - **`error` string:** Single human-readable reason, used by the older endpoints in place of `errors`. Only present when `success` is false. PUT `/v1/lab/{id}/report` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl -X PUT "https://api.testingbot.com/v1/lab/{id}/report" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); await client.CodelessTests.UpdateReportAsync(testId, new CodelessReportInput { Email = "ops@example.com" }); ``` ```php $client = new TestingBot\Client($key, $secret); $client->lab()->updateReport($labTestId, ['email' => 'ops@example.com']); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); Map report = Map.of("email", "ops@example.com"); boolean success = restApi.updateLabTestReport(labTestId, report); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); await api.updateCodelessReport(testId, { email: 'ops@example.com' }); ``` GET `/v1/jobs/{id}` ## Get a job's status Returns the live status of an asynchronous job — typically used to poll Codeless test/suite runs started by `POST /v1/lab/:id/trigger`, `POST /v1/lab/trigger_all`, or `POST /v1/labsuites/:id/trigger`. Once `status` is FINISHED, the response includes `success` and per-test results. ### Arguments - **`id` integer required:** Numeric job ID returned by a trigger endpoint. ### Response fields - **`status` string:** Job state (QUEUED, RUNNING, FINISHED, FAILED). - **`created_at` timestamp:** — - **`updated_at` timestamp:** — - **`success` boolean:** Aggregate pass/fail across all triggered tests; null until FINISHED. - **`test_ids` array of integer:** Test IDs spawned by this job. - **`errors` array of object:** Per-test failure detail (step, browser, time). GET `/v1/jobs/{id}` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/jobs/{id}" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var job = await client.Jobs.GetAsync(jobId); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_job(job_id) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.jobs.get_job(job_id) ``` ```php $client = new TestingBot\Client($key, $secret); $job = $client->jobs()->get($jobId); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingbotJob job = restApi.getJob(jobId); ``` Response ```json { "status": "FINISHED", "created_at": "2016-04-15T13:47:39.000Z", "updated_at": "2016-04-15T13:49:30.000Z", "success": false, "test_ids": [6620446], "errors": [ { "msg": "Actual value 'Google' did not match 'Goooogle'", "step": "verifyTitle", "browser": { "name": "firefox", "version": "43", "os": "VISTA" }, "time": "2016-04-15T13:48:25.839Z", "test": 6620446 } ] } ``` POST `(your callback URL)` ## Webhook callback If you have configured a webhook URL, we will POST the result to the callback URL you configured in the Codeless alerts section. The body below is the JSON payload we send; respond with 2xx to acknowledge. POST `(your callback URL)` Response ```json { "success": false, "errors": [ { "msg": "word Not found", "step": "verifyTextPresent", "browser": { "name": "iexplore", "version": "8", "os": "WINDOWS" }, "time": "2012-03-13 20:34:59 UTC", "test_id": "48586", "job_id": 17, "lab_id": 133 } ], "job_id": 3, "test_ids": [48586] } ``` GET `/v1/labsuites` ## List your Codeless suites Paginated list of every Codeless suite (a.k.a. Lab suite) on the account. A suite groups several Codeless tests so they can be scheduled, triggered, and reported on as a unit. ### Arguments - **`offset` integer:** Skip this many suites from the start of the result set. - **`count` integer max=`500`:** Number of suites to return . ### Response fields - **`data` array of lab suite objects:** — - **`meta` meta object:** — GET `/v1/labsuites` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/labsuites" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var suites = await client.CodelessSuites.ListAsync(); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_lab_suites(0, 10) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.labsuites.get_suites(offset=0, limit=10) ``` ```php $client = new TestingBot\Client($key, $secret); $suites = $client->labSuites()->list(0, 10); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingbotLabSuiteCollection suites = restApi.getLabSuites(); ``` Response ```json { "data": [ { "id": 1051, "enabled": true, "name": "Example", "created_at": "2017-03-07T19:11:41.000Z", "updated_at": "2017-03-09T12:17:22.000Z", "last_run": "2017-03-09T12:17:22.000Z", "cron": null, "test_count": 5, "alerts": [ { "type": "API", "value": "https://mysite.com/callback/", "level": "IMMEDIATELY" } ], "browsers": [ { "name": "firefox", "version": "41", "os": "VISTA" } ] } ], "meta": { "offset": 0, "count": 10, "total": 1 } } ``` GET `/v1/labsuites/{id}` ## Get a specific Codeless suite Returns a single Codeless suite's configuration: schedule, alerts, attached browsers, and number of tests inside. ### Arguments - **`id` integer required:** Numeric suite ID. ### Response fields - **`id` integer:** Unique numeric Codeless suite ID. - **`name` string:** Suite name. - **`enabled` boolean:** Whether scheduled runs are active. - **`cron` string:** Cron expression for scheduled runs. - **`test_count` integer:** Number of Codeless tests attached. - **`created_at` timestamp:** — - **`updated_at` timestamp:** — - **`last_run` timestamp:** — - **`alerts` array of lab alert objects:** — - **`browsers` array of lab browser objects:** Browsers the suite runs against. Only present on GET /v1/labsuites; the single-suite endpoint omits it. GET `/v1/labsuites/{id}` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/labsuites/{id}" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var suite = await client.CodelessSuites.GetAsync(suiteId); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_lab_suite(suite_id) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.labsuites.get_suite(suite_id) ``` ```php $client = new TestingBot\Client($key, $secret); $suite = $client->labSuites()->get($suiteId); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingbotLabSuite suite = restApi.getLabSuite(suiteId); ``` Response ```json { "id": 41, "enabled": true, "name": "Example", "cron": null, "test_count": 5, "created_at": "2017-03-07T19:11:41.000Z", "updated_at": "2017-03-09T12:17:22.000Z", "last_run": "2017-03-09T12:17:22.000Z", "alerts": [ { "type": "API", "value": "https://mysite.com/callback/", "level": "IMMEDIATELY" } ] } ``` POST `/v1/labsuites/{id}/trigger` ## Run a Codeless suite Queues every test in the suite for an immediate run. Returns a job\_id you can poll with `GET /v1/jobs/:id`. ### Arguments - **`id` integer required:** Numeric suite ID to trigger. POST `/v1/labsuites/{id}/trigger` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/labsuites/{id}/trigger" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var result = await client.CodelessSuites.TriggerAsync(suiteId); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.trigger_lab_suite(suite_id) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.labsuites.trigger(suite_id) ``` ```php $client = new TestingBot\Client($key, $secret); $run = $client->labSuites()->trigger($suiteId); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingbotLabRunAck ack = restApi.triggerLabSuite(suiteId); ``` Response ```json { "success": true, "job_id": 14 } ``` POST `/v1/labsuites` ## Create a Codeless suite Creates a new Codeless suite. Attach Codeless tests with `POST /v1/labsuites/:id/tests` after creation. ### Arguments - **`suite[name]` string required:** Suite name. - **`suite[cron]` string:** Cron expression for scheduled suite runs. - **`suite[screenshot]` boolean:** Take screenshots at every step in every test. - **`suite[video]` boolean:** Record video for tests in this suite. - **`suite[idletimeout]` integer:** Idle timeout in seconds before tests are aborted. - **`suite[screenresolution]` string:** Browser viewport for every test in the suite. POST `/v1/labsuites` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/labsuites" \ -u key:secret \ -d "suite[name]=My Suite" ``` ```csharp var client = new TestingBotClient(key, secret); long suiteId = await client.CodelessSuites.CreateAsync(new CodelessSuiteCreate { Name = "Regression suite" }); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.create_lab_suite({ "suite[name]" => 'My Suite' }) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.labsuites.create_suite('My Suite') ``` ```php $client = new TestingBot\Client($key, $secret); $suite = $client->labSuites()->create(['name' => 'regression']); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); Map fields = new HashMap<>(); fields.put("name", "My Suite"); TestingbotLabSuiteCreateAck ack = restApi.createLabSuite(fields); ``` Response ```json { "success": true, "suite_id": 59391 } ``` DELETE `/v1/labsuites/{id}` ## Delete a Codeless suite Deletes the suite. The Codeless tests attached to it are not deleted — only the suite grouping. ### Arguments - **`id` integer required:** Numeric suite ID to delete. DELETE `/v1/labsuites/{id}` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) Request ```bash $ curl -X DELETE "https://api.testingbot.com/v1/labsuites/{id}" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); await client.CodelessSuites.DeleteAsync(suiteId); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.delete_lab_suite(suite_id) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.labsuites.delete_suite(suite_id) ``` ```php $client = new TestingBot\Client($key, $secret); $client->labSuites()->delete($suiteId); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); boolean success = restApi.deleteLabSuite(suiteId); ``` Response ```json { "success": true } ``` GET `/v1/labsuites/{id}/browsers` ## Get browsers for a Codeless suite Returns the list of browsers the suite is configured to run on. ### Arguments - **`id` integer required:** Numeric suite ID. ### Response fields - **`name` string:** Browser identifier (e.g. "firefox", "chrome"). - **`version` string:** Browser version the test runs against. - **`os` string:** Operating system the test runs on (e.g. "WINDOWS", "MAC", "LINUX"). GET `/v1/labsuites/{id}/browsers` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/labsuites/{id}/browsers" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var browsers = await client.CodelessSuites.GetBrowsersAsync(suiteId); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_lab_suite_browsers(suite_id) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.labsuites.get_browsers(suite_id) ``` ```php $client = new TestingBot\Client($key, $secret); $browsers = $client->labSuites()->getBrowsers($suiteId); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); List browsers = restApi.getLabSuiteBrowsers(suiteId); ``` Response ```json [ { "name": "firefox", "version": "41", "os": "VISTA" } ] ``` POST `/v1/labsuites/{id}/browsers` ## Update browsers for a Codeless suite Replaces the browser set attached to a suite. Every test in the suite will run on the new browser list at next trigger. ### Arguments - **`id` integer required:** Numeric suite ID. - **`browser_ids` string required:** Comma-separated list of browser\_ids the suite should run on. POST `/v1/labsuites/{id}/browsers` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/labsuites/{id}/browsers" \ -u key:secret \ -d "browser_ids=1,5,12" ``` ```csharp var client = new TestingBotClient(key, secret); await client.CodelessSuites.SetBrowsersAsync(suiteId, new[] { 1, 22 }); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.set_lab_suite_browsers(suite_id, { browser_ids: '1,5,12' }) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.labsuites.set_browsers(suite_id, '1,5,12') ``` ```php $client = new TestingBot\Client($key, $secret); $client->labSuites()->setBrowsers($suiteId, [1, 2]); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); boolean success = restApi.setLabSuiteBrowsers(suiteId, "1,5,12"); ``` Response ```json { "success": true } ``` GET `/v1/labsuites/{id}/tests` ## List tests in a Codeless suite Returns the Codeless tests attached to a suite, with pagination. ### Arguments - **`id` integer required:** Numeric suite ID. - **`offset` integer:** Skip this many tests. - **`count` integer:** Number of tests to return. ### Response fields - **`data` array of lab test objects:** — - **`meta` meta object:** — GET `/v1/labsuites/{id}/tests` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/labsuites/{id}/tests" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var tests = await client.CodelessSuites.GetTestsAsync(suiteId); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_lab_suite_tests(suite_id) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.labsuites.get_tests(suite_id) ``` ```php $client = new TestingBot\Client($key, $secret); $tests = $client->labSuites()->getTests($suiteId, 0, 10); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingbotLabTestCollection tests = restApi.getLabSuiteTests(suiteId); ``` Response ```json { "data": [ { "id": 18666, "enabled": true, "name": "MyTest", "url": "https://mysite.com/", "created_at": "2017-03-09T12:14:48.000Z", "updated_at": "2017-03-09T12:19:30.000Z", "last_run": "2017-03-09T12:19:30.000Z", "cron": null, "browsers": [ { "name": "firefox", "version": "41", "os": "VISTA" } ] } ], "meta": { "offset": 0, "count": 1, "total": 1 } } ``` POST `/v1/labsuites/{id}/tests` ## Add tests to a Codeless suite Attaches one or more existing Codeless tests to a suite. Pass `test_ids` as a comma-separated list. ### Arguments - **`id` integer required:** Numeric suite ID. - **`test_ids` string required:** Comma-separated list of Codeless test IDs to attach. POST `/v1/labsuites/{id}/tests` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/labsuites/{id}/tests" \ -u key:secret \ -d "test_ids=215,228" ``` ```csharp var client = new TestingBotClient(key, secret); await client.CodelessSuites.AddTestsAsync(suiteId, new long[] { 123, 456 }); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.add_lab_suite_tests(suite_id, { test_ids: '215,228' }) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.labsuites.add_tests(suite_id, '215,228') ``` ```php $client = new TestingBot\Client($key, $secret); $client->labSuites()->addTests($suiteId, [$labTestId]); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); boolean success = restApi.addLabSuiteTests(suiteId, "215,228"); ``` Response ```json { "success": true } ``` DELETE `/v1/labsuites/{id}/tests/{testid}` ## Remove a test from a Codeless suite Detaches a single Codeless test from a suite. The test itself is preserved. ### Arguments - **`id` integer required:** Numeric suite ID. - **`testid` integer required:** Numeric Codeless test ID to detach. DELETE `/v1/labsuites/{id}/tests/{testid}` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) Request ```bash $ curl -X DELETE "https://api.testingbot.com/v1/labsuites/{id}/tests/{testid}" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); await client.CodelessSuites.RemoveTestAsync(suiteId, testId); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.remove_lab_suite_test(suite_id, test_id) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.labsuites.remove_test(suite_id, test_id) ``` ```php $client = new TestingBot\Client($key, $secret); $client->labSuites()->removeTest($suiteId, $labTestId); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); boolean success = restApi.removeLabSuiteTest(suiteId, testId); ``` Response ```json { "success": true } ``` [Previous Tunnel](https://testingbot.com/support/api/tunnel) [Next TestingBot Storage](https://testingbot.com/support/api/storage) --- URL: https://testingbot.com/support/api/storage # TestingBot Storage Upload app binaries and test bundles once, then reference them from your capabilities with a tb:// URL. - **Endpoint:** api.testingbot.com - **Version:** v1 - **Format:** JSON - **Auth:** [HTTP Basic](https://testingbot.com/support/api#authentication) POST `/v1/storage` ## Upload an app to storage Uploads an APK or IPA to TestingBot storage. Either send the binary as multipart `file` or pass a public `url` that TestingBot will fetch. The returned `app_url` (`tb://`) can be set as the `app` capability on subsequent mobile sessions. Metadata (version, min\_device\_version, thumb) is extracted asynchronously: poll GET /storage/\ until its `state` is `DONE` before reading those fields. ### Arguments - **`file` file:** Multipart binary upload of an .apk or .ipa. - **`url` string:** Public HTTPS URL TestingBot will download the binary from. POST `/v1/storage` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/storage" \ -u key:secret \ -F "file=@/path/to/app/file/Application-debug.apk" ``` ```csharp var client = new TestingBotClient(key, secret); var file = await client.Storage.UploadAsync(new FileInfo("/path/to/Application-debug.apk")); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.upload_local_file(local_file_path) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.storage.upload_local_file(local_file_path) ``` ```php $api = new TestingBot\TestingBotAPI($key, $secret); $api->uploadLocalFileToStorage($pathToLocalFile); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingbotStorageUploadResponse uploadResponse = restApi.uploadToStorage(file); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const { app_url } = await api.uploadFile(localFilePath); ``` Response ```json { "app_url": "tb://398eijf83i" } ``` POST `/v1/storage/{app_url}` ## Update an app binary by appkey Replaces the binary stored under an existing `appkey`. The `app_url` (`tb://`) stays the same so deployed CI configurations keep working without changes — useful for "always use the latest" CI flows. Metadata is re-extracted asynchronously: poll GET /storage/\ until its `state` is `DONE`. ### Arguments - **`app_url` string required:** The app to operate on: its `appkey`, or the full `tb://` app\_url. - **`file` file:** Multipart binary replacement. - **`url` string:** Public HTTPS URL to fetch the new binary from. POST `/v1/storage/{app_url}` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/storage/{app_url}" \ -u key:secret \ -F "file=@/path/to/app/file/Application-debug.apk" ``` ```csharp var client = new TestingBotClient(key, secret); var file = await client.Storage.ReplaceAsync(appKey, new FileInfo("/path/to/Application-debug.apk")); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.update_local_file(app_url, local_file_path) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.storage.replace_local_file(app_url, local_file_path) ``` ```php $client = new TestingBot\Client($key, $secret); $file = $client->storage()->replaceFile($appUrl, '/path/to/Application-debug.apk'); ``` Response ```json { "app_url": "tb://398eijf83i" } ``` GET `/v1/storage/{app_url}` ## Get a stored app by appkey Returns metadata for a single uploaded app — its filename, type, version, icon, and download URL. ### Arguments - **`app_url` string required:** The app to operate on: its `appkey`, or the full `tb://` app\_url. ### Response fields - **`id` integer:** Unique numeric storage object ID. - **`app_url` string:** TestingBot storage URL (`tb://`) — pass as a capability to reference this app in mobile tests. - **`url` string:** Signed HTTPS URL to download the binary directly. - **`filename` string:** Original uploaded filename. - **`type` string:** Detected app type (ANDROID for .apk/.aab, IOS for .ipa/.zip, OTHER otherwise). - **`version` string:** CFBundleShortVersionString / versionName extracted from the binary. - **`min_device_version` string:** Minimum OS version required by the app. - **`thumb` string:** App icon (signed PNG URL). - **`created_at` timestamp:** Upload timestamp. - **`state` string:** Metadata extraction state: PROCESSING right after upload, DONE once version / min\_device\_version / thumb are populated. Poll until DONE before relying on those fields. - **`sim_only` boolean:** True for iOS simulator-only IPAs (no signing). GET `/v1/storage/{app_url}` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/storage/{app_url}" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var file = await client.Storage.GetAsync(appUrlOrKey); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_uploaded_file(app_url) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.storage.get_stored_file(app_url) ``` ```php $api = new TestingBot\TestingBotAPI($key, $secret); $api->getStorageFile($appUrl); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingBotStorageFile storedFile = restApi.getStorageFile(appUrl); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); await api.getStorageFile(appUrl); ``` Response ```json { "app_url": "tb://0ec522702cd81ec1374e9b3c", "url": "https://...", "id": 261, "type": "ANDROID", "filename": "sample.apk", "version": "1.0.1", "min_device_version": "12.0", "thumb": "https://...image.png", "created_at": "2019-03-08T08:58:32.000Z", "state": "DONE", "sim_only": false } ``` GET `/v1/storage` ## List your storage apps Paginated list of every app you've uploaded to TestingBot storage, newest first. ### Arguments - **`offset` integer:** Skip this many apps from the start of the result set. - **`count` integer max=`500`:** Number of apps to return . ### Response fields - **`data` array of storage file objects:** Storage objects for this page. - **`meta` meta object:** — GET `/v1/storage` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/storage/" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); var files = await client.Storage.ListAsync(); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_uploaded_files(0, 30) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.storage.get_stored_files(offset=0, limit=30) ``` ```php $api = new TestingBot\TestingBotAPI($key, $secret); $api->getStorageFiles(); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); TestingBotStorageFileCollection fileList = restApi.getStorageFiles(0, 30); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const files = await api.getStorageFiles(offset, count); ``` Response ```json { "data": [ { "app_url": "tb://….", "url": "…", "id": 44, "type": "ANDROID", "filename": "sample.apk", "version": "1.0.1", "min_device_version": "23", "thumb": "https://...image.png", "created_at": "2019-01-08T13:42:42.000Z", "state": "DONE", "sim_only": false } ], "meta": { "offset": 0, "count": 10, "total": 1 } } ``` DELETE `/v1/storage/{id}` ## Delete a stored app Permanently removes the binary and its metadata. Sessions referencing the `app_url` after deletion will fail at session-start with "app not found". ### Arguments - **`id` string required:** Numeric storage ID or appkey to delete. DELETE `/v1/storage/{id}` [cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) Request ```bash $ curl -X DELETE "https://api.testingbot.com/v1/storage/{app_url}" \ -u key:secret ``` ```csharp var client = new TestingBotClient(key, secret); await client.Storage.DeleteAsync(idOrAppKey); ``` ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.delete_uploaded_file(file_identifier) ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.storage.remove_file(app_url) ``` ```php $api = new TestingBot\TestingBotAPI($key, $secret); $api->deleteStorageFile($appUrl); ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); boolean success = restApi.deleteStorageFile(appUrl); ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const success = await api.deleteStorageFile(fileId); ``` Response ```json { "success": true } ``` [Previous Codeless Automation & TestLab](https://testingbot.com/support/api/codeless) [Next Webhooks](https://testingbot.com/support/api/webhooks) --- URL: https://testingbot.com/support/api/webhooks # Webhooks Manage the callbacks TestingBot posts when a test finishes, and verify them without waiting for a run. - **Endpoint:** api.testingbot.com - **Version:** v1 - **Format:** JSON - **Auth:** [HTTP Basic](https://testingbot.com/support/api#authentication) GET `/v1/webhooks` ## List your webhooks Returns every webhook on the team. Webhooks fire when a test finishes; use `failure_only`, `alert_types` and `name_filter` to narrow which tests trigger a delivery. The signing secret is never returned here — it is shown once when the webhook is created or its secret is rotated. ### Arguments - **`offset` integer:** Skip this many webhooks from the start of the result set. - **`count` integer:** Number of webhooks to return. ### Response fields - **`data` array of webhook objects:** Webhooks on this team. - **`meta` meta object:** — GET `/v1/webhooks` [cURL](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/webhooks?offset=0&count=10" \ -u key:secret ``` Response ```json { "data": [ { "id": 50, "name": "CI notifier", "url": "https://example.com/hooks/testingbot", "http_method": "POST", "failure_only": true, "alert_types": "ALL", "name_filter": null, "auth_type": "NONE", "headers": [{ "key": "X-Source", "value": "testingbot" }], "params": [], "payload_template": null, "created_at": "2026-08-03T10:00:32.000Z", "updated_at": "2026-08-03T10:00:32.000Z" } ], "meta": { "offset": 0, "count": 10, "total": 1 } } ``` GET `/v1/webhooks/{id}` ## Get a specific webhook Returns a single webhook by ID. Receiver credentials and the signing secret are never included. ### Arguments - **`id` integer required:** Numeric webhook ID. ### Response fields - **`id` integer:** Unique numeric webhook ID. - **`name` string:** Label for the webhook, shown on the dashboard. - **`url` string:** Destination URL. May contain `{{VARIABLES}}`, which are substituted at delivery time. - **`http_method` string:** HTTP method used to deliver: POST, PUT or PATCH. - **`failure_only` boolean:** When true, only failed tests trigger a delivery. - **`alert_types` string:** Which grid the test ran on: ALL, VIRTUAL (VM) or REAL (physical device). - **`name_filter` string:** Glob matched against the test name, falling back to the build identifier. Empty means every test. - **`auth_type` string:** How the request authenticates against your endpoint: NONE, BASIC or BEARER. The credentials themselves are never returned. - **`headers` array of object:** Custom request headers, each `{ "key": ..., "value": ... }`. Values support `{{VARIABLES}}`. - **`params` array of object:** Custom query parameters appended to the URL, each `{ "key": ..., "value": ... }`. - **`payload_template` string:** Custom JSON body template. Empty means the default TestingBot payload is sent. - **`created_at` timestamp:** When the webhook was created. - **`updated_at` timestamp:** When the webhook was last changed. - **`signing_secret` string:** Secret used to verify the `X-TestingBot-Signature` header. Only present when the webhook is created or its secret is rotated; store it then, because it is never returned again. GET `/v1/webhooks/{id}` [cURL](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/webhooks/{id}" \ -u key:secret ``` Response ```json { "id": 50, "name": "CI notifier", "url": "https://example.com/hooks/testingbot", "http_method": "POST", "failure_only": true, "alert_types": "ALL", "name_filter": null, "auth_type": "NONE", "headers": [{ "key": "X-Source", "value": "testingbot" }], "params": [], "payload_template": null, "created_at": "2026-08-03T10:00:32.000Z", "updated_at": "2026-08-03T10:00:32.000Z" } ``` POST `/v1/webhooks` ## Create a webhook Creates a webhook. The response includes `signing_secret` — the only time it is ever returned — which you use to verify the `X-TestingBot-Signature` header on deliveries. A team may have at most 5 webhooks. ### Arguments - **`url` string required:** Destination URL. Must be a public http(s) address; private, loopback and link-local hosts are rejected. - **`name` string:** Label for the webhook. Required once the extended webhook columns are present. - **`http_method` string:** HTTP method used to deliver. Defaults to POST. - **`failure_only` boolean:** Deliver only for failed tests. Defaults to false. - **`alert_types` string:** Which grid to report on: ALL, VIRTUAL or REAL. - **`name_filter` string:** Glob matched against the test name, falling back to the build identifier. - **`auth_type` string:** How to authenticate against your endpoint: NONE, BASIC or BEARER. - **`auth_username` string:** Username for BASIC auth. Write-only, never returned. - **`auth_password` string:** Password for BASIC auth. Write-only, never returned. - **`auth_token` string:** Token for BEARER auth. Write-only, never returned. - **`headers` array:** Custom request headers, each an object with `key` and `value`. - **`params` array:** Custom query parameters appended to the URL, each an object with `key` and `value`. - **`payload_template` string:** Custom JSON body template using `{{VARIABLES}}`. Omit to send the default TestingBot payload. POST `/v1/webhooks` [cURL](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/webhooks" \ -u key:secret \ -H "Content-Type: application/json" \ -d '{ "name": "CI notifier", "url": "https://example.com/hooks/testingbot", "failure_only": true, "alert_types": "ALL", "headers": [{ "key": "X-Source", "value": "testingbot" }] }' ``` Response ```json { "id": 50, "name": "CI notifier", "url": "https://example.com/hooks/testingbot", "http_method": "POST", "failure_only": true, "alert_types": "ALL", "name_filter": null, "auth_type": "NONE", "headers": [{ "key": "X-Source", "value": "testingbot" }], "params": [], "payload_template": null, "created_at": "2026-08-03T10:00:32.000Z", "updated_at": "2026-08-03T10:00:32.000Z", "signing_secret": "whsec_a1b2c3d4e5f60718293a4b5c6d7e8f90" } ``` PUT `/v1/webhooks/{id}` ## Update a webhook Updates a webhook. Only the fields you send are changed. Sending `auth_password` or `auth_token` replaces the stored value; omitting them leaves it alone. The signing secret is not returned — rotate it if you have lost it. ### Arguments - **`id` integer required:** Numeric webhook ID. - **`url` string:** Destination URL. Must be a public http(s) address. - **`name` string:** Label for the webhook. - **`http_method` string:** HTTP method used to deliver. - **`failure_only` boolean:** Deliver only for failed tests. - **`alert_types` string:** Which grid to report on: ALL, VIRTUAL or REAL. - **`name_filter` string:** Glob matched against the test name, falling back to the build identifier. - **`auth_type` string:** How to authenticate against your endpoint. - **`auth_username` string:** Username for BASIC auth. Write-only. - **`auth_password` string:** Password for BASIC auth. Write-only. - **`auth_token` string:** Token for BEARER auth. Write-only. - **`headers` array:** Custom request headers, each an object with `key` and `value`. Replaces the existing set. - **`params` array:** Custom query parameters, each an object with `key` and `value`. Replaces the existing set. - **`payload_template` string:** Custom JSON body template. Send an empty string to go back to the default payload. ### Response fields - **`id` integer:** Unique numeric webhook ID. - **`name` string:** Label for the webhook, shown on the dashboard. - **`url` string:** Destination URL. May contain `{{VARIABLES}}`, which are substituted at delivery time. - **`http_method` string:** HTTP method used to deliver: POST, PUT or PATCH. - **`failure_only` boolean:** When true, only failed tests trigger a delivery. - **`alert_types` string:** Which grid the test ran on: ALL, VIRTUAL (VM) or REAL (physical device). - **`name_filter` string:** Glob matched against the test name, falling back to the build identifier. Empty means every test. - **`auth_type` string:** How the request authenticates against your endpoint: NONE, BASIC or BEARER. The credentials themselves are never returned. - **`headers` array of object:** Custom request headers, each `{ "key": ..., "value": ... }`. Values support `{{VARIABLES}}`. - **`params` array of object:** Custom query parameters appended to the URL, each `{ "key": ..., "value": ... }`. - **`payload_template` string:** Custom JSON body template. Empty means the default TestingBot payload is sent. - **`created_at` timestamp:** When the webhook was created. - **`updated_at` timestamp:** When the webhook was last changed. - **`signing_secret` string:** Secret used to verify the `X-TestingBot-Signature` header. Only present when the webhook is created or its secret is rotated; store it then, because it is never returned again. PUT `/v1/webhooks/{id}` [cURL](https://testingbot.com#) Request ```bash $ curl -X PUT "https://api.testingbot.com/v1/webhooks/{id}" \ -u key:secret \ -H "Content-Type: application/json" \ -d '{ "name": "CI notifier (failures only)", "failure_only": true }' ``` Response ```json { "id": 50, "name": "CI notifier (failures only)", "url": "https://example.com/hooks/testingbot", "http_method": "POST", "failure_only": true, "alert_types": "ALL", "name_filter": null, "auth_type": "NONE", "headers": [{ "key": "X-Source", "value": "testingbot" }], "params": [], "payload_template": null, "created_at": "2026-08-03T10:00:32.000Z", "updated_at": "2026-08-03T10:04:11.000Z" } ``` DELETE `/v1/webhooks/{id}` ## Delete a webhook Permanently deletes the webhook. Deliveries stop immediately; tests already in flight will not fire it. ### Arguments - **`id` integer required:** Numeric webhook ID. DELETE `/v1/webhooks/{id}` [cURL](https://testingbot.com#) Request ```bash $ curl -X DELETE "https://api.testingbot.com/v1/webhooks/{id}" \ -u key:secret ``` Response ```json { "success": true, "message": "Webhook \"https://example.com/hooks/testingbot\" deleted." } ``` POST `/v1/webhooks/{id}/rotate-secret` ## Rotate a webhook signing secret Issues a new signing secret and returns it. The previous secret stops verifying immediately, so update your receiver before rotating in production. This is the only other time the secret is returned. ### Arguments - **`id` integer required:** Numeric webhook ID. POST `/v1/webhooks/{id}/rotate-secret` [cURL](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/webhooks/{id}/rotate-secret" \ -u key:secret ``` Response ```json { "id": 50, "signing_secret": "whsec_f0e9d8c7b6a5948372615f4e3d2c1b0a" } ``` POST `/v1/webhooks/{id}/test` ## Send a test delivery Delivers a sample payload to the webhook right now and returns what your endpoint responded, so you can verify the URL, authentication and signature verification without waiting for a test to finish. Sample data is used, not a real test result. Rate limited separately from the rest of the API. ### Arguments - **`id` integer required:** Numeric webhook ID. POST `/v1/webhooks/{id}/test` [cURL](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/webhooks/{id}/test" \ -u key:secret ``` Response ```json { "ok": true, "status": 200, "duration_ms": 148, "headers": { "content-type": "application/json" }, "body": "{\"received\":true}", "sent_payload": "{\"id\":1,\"status\":\"FAILED\",\"name\":\"MyTest::testTitle\"}" } ``` [Previous TestingBot Storage](https://testingbot.com/support/api/storage) [Next App Automate](https://testingbot.com/support/api/app-automate) --- URL: https://testingbot.com/support/api/app-automate # App Automate Everything for native mobile test runs: Maestro flows, Espresso and XCUITest suites, and their reports. - **Endpoint:** api.testingbot.com - **Version:** v1 - **Format:** JSON - **Auth:** [HTTP Basic](https://testingbot.com/support/api#authentication) ## App Automate: Espresso POST `/v1/app-automate/espresso/app` ## Upload an Android app for Espresso Uploads the production `.apk` you want to test under Espresso. Returns the `app_url` to reference when starting a run. POST `/v1/app-automate/espresso/app` [cURL](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/app-automate/espresso/app" \ -u key:secret ``` GET `/v1/app-automate/espresso/{id}` ## Get an Espresso project Returns the metadata and current state of an Espresso project. ### Arguments - **`id` integer required:** — ### Response fields - **`runs` array of object:** Runs belonging to the project. - **`success` boolean:** True when every run in the project passed. - **`completed` boolean:** True when every run has reached a terminal state. GET `/v1/app-automate/espresso/{id}` [cURL](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/app-automate/espresso/{id}" \ -u key:secret ``` GET `/v1/app-automate/espresso/{id}/report` ## Get the Espresso JUnit report Returns the JUnit-format XML report for an Espresso run once it has completed. ### Arguments - **`id` integer required:** — GET `/v1/app-automate/espresso/{id}/report` [cURL](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/app-automate/espresso/{id}/report" \ -u key:secret ``` POST `/v1/app-automate/espresso/{id}/run` ## Run an Espresso test suite Triggers an asynchronous Espresso run across the requested device capabilities. Returns run metadata; poll status via the standard tests API. ### Arguments - **`id` integer required:** — POST `/v1/app-automate/espresso/{id}/run` [cURL](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/app-automate/espresso/{id}/run" \ -u key:secret ``` POST `/v1/app-automate/espresso/{id}/tests` ## Upload Espresso test APK Attaches an Espresso test `.apk` (the instrumented test bundle) to a project. Pair with `/app` to provide both halves of the test. ### Arguments - **`id` integer required:** — POST `/v1/app-automate/espresso/{id}/tests` [cURL](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/app-automate/espresso/{id}/tests" \ -u key:secret ``` GET `/v1/app-automate/espresso/{project_id}/{id}` ## Get a specific Espresso run Returns the detail record for a single Espresso run inside a project. ### Arguments - **`project_id` integer required:** — - **`id` integer required:** — ### Response fields - **`id` integer:** Unique numeric Espresso run ID. - **`created_at` timestamp:** — - **`status` string:** WAITING, READY, DONE or FAILED. - **`capabilities` object:** Capabilities the run was started with. - **`success` boolean:** Whether the run passed. False until a session has completed. - **`report` object:** Test counts and per-test results for the run. - **`test` object:** Session detail. Only present once a session has been allocated. GET `/v1/app-automate/espresso/{project_id}/{id}` [cURL](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/app-automate/espresso/{project_id}/{id}" \ -u key:secret ``` ## App Automate: Maestro GET `/v1/app-automate/maestro` ## List your Maestro projects Paginated list of Maestro app-automate projects (one per uploaded test bundle) on the account, newest first. ### Response fields - **`data` array of maestro project objects:** Projects for this page, newest first. - **`meta` meta object:** — GET `/v1/app-automate/maestro` [cURL](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/app-automate/maestro" \ -u key:secret ``` POST `/v1/app-automate/maestro/app` ## Upload an Android/iOS app for Maestro Uploads an `.apk` / `.ipa` to attach to a Maestro project. Returns the `app_url` (`tb://`) you'll pass as a capability when running the project. ### Arguments - **`file` string required:** — - **`checksum` string:** MD5 checksum (base64) of the app file for deduplication POST `/v1/app-automate/maestro/app` [cURL](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/app-automate/maestro/app" \ -u key:secret \ -F "file=@/path/to/app/file/Application-debug.apk" ``` POST `/v1/app-automate/maestro/app/checksum` ## Check whether an app needs uploading Returns whether a binary identified by checksum already exists in TestingBot Storage. Used by the CLI to skip re-uploading unchanged apps. ### Arguments - **`checksum` string required:** MD5 checksum (base64) of the app file POST `/v1/app-automate/maestro/app/checksum` [cURL](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/app-automate/maestro/app/checksum" \ -u key:secret \ -d "checksum=$(openssl md5 -binary Application-debug.apk | base64)" ``` POST `/v1/app-automate/maestro/other-apps` ## Upload additional apps for a Maestro session Uploads supplementary `.apk`/`.ipa` binaries that should be installed alongside the primary app under test (e.g. dependent apps, test runners). ### Arguments - **`file` string required:** — POST `/v1/app-automate/maestro/other-apps` [cURL](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/app-automate/maestro/other-apps" \ -u key:secret \ -F "file=@/path/to/app/file/DependencyApp-debug.apk" ``` GET `/v1/app-automate/maestro/runs` ## List your Maestro runs Paginated list of Maestro app-automate runs across all projects on the account, newest first. Lightweight: no reports are inflated. Use GET /:project\_id/:id for a single run's full results. ### Response fields - **`data` array of maestro run summary objects:** Runs for this page, newest first. - **`meta` meta object:** — GET `/v1/app-automate/maestro/runs` [cURL](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/app-automate/maestro/runs" \ -u key:secret ``` GET `/v1/app-automate/maestro/runs/{build_name}` ## Look up Maestro runs by build name Resolves a build/run name to every matching run on the account (a build can fan out to multiple device runs). Returns project\_id + run\_id without inflating reports. Use GET /:project\_id/:id for a single run's full results. ### Arguments - **`build_name` string required:** — ### Response fields - **`data` array of maestro run summary objects:** Every run submitted under this build name, newest first. GET `/v1/app-automate/maestro/runs/{build_name}` [cURL](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/app-automate/maestro/runs/{build_name}" \ -u key:secret ``` GET `/v1/app-automate/maestro/{id}` ## Get a Maestro project Returns the metadata and current state of a single Maestro app-automate project. ### Arguments - **`id` integer required:** — ### Response fields - **`runs` array of object:** Runs belonging to the project. - **`success` boolean:** True when every run in the project passed. - **`completed` boolean:** True when every run has reached a terminal state. GET `/v1/app-automate/maestro/{id}` [cURL](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/app-automate/maestro/{id}" \ -u key:secret ``` POST `/v1/app-automate/maestro/{id}/run` ## Run a Maestro app flow Triggers an asynchronous Maestro run on the uploaded project against one or more device capabilities. Returns one run entry per capability. ### Arguments - **`id` integer required:** — POST `/v1/app-automate/maestro/{id}/run` [cURL](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/app-automate/maestro/{id}/run" \ -u key:secret ``` POST `/v1/app-automate/maestro/{id}/tests` ## Upload a Maestro test bundle Attaches a Maestro flow archive (`.zip` or `.yaml`) to an existing Maestro project. Replaces any previously uploaded tests for that project. ### Arguments - **`id` integer required:** — POST `/v1/app-automate/maestro/{id}/tests` [cURL](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/app-automate/maestro/{id}/tests" \ -u key:secret ``` GET `/v1/app-automate/maestro/{project_id}/{id}` ## Get a specific Maestro run Returns the detail record for a single Maestro run inside a project, including capability used, status, and per-flow results. ### Arguments - **`project_id` integer required:** — - **`id` integer required:** — ### Response fields - **`id` integer:** Unique numeric Maestro run ID. - **`status` string:** WAITING, READY, DONE, FAILED or CANCELLED, computed from the newest attempt of each flow. - **`success` integer:** 1 when every flow passed, 0 otherwise. - **`capabilities` object:** Capabilities the run was started with. - **`options` object:** Maestro options the run was started with; `{}` when none were given. - **`created_at` timestamp:** — - **`completed` boolean:** Whether every run in the parent project has reached a terminal state. - **`assets_synced` boolean:** Whether asset processing has finished for every flow in the run. - **`meta` object:** Metadata supplied at submission. Only present when the run carries any. - **`report` string:** JUnit XML for the run. Only present once a report exists. - **`test` object:** Session detail. Only present once a session has been allocated. - **`assets` object:** Logs, video and screenshots. Only present once asset processing has finished. - **`environment` object:** Device the run was allocated. Only present once one has been assigned. - **`flows` array of maestro flow result objects:** Every flow attempt, including retries. Only present once the run has flows. - **`error_messages` array of string:** Run-level failures from the newest attempt of each flow, so a failure a retry has since fixed does not linger. Only present when the latest attempts reported any. GET `/v1/app-automate/maestro/{project_id}/{id}` [cURL](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/app-automate/maestro/{project_id}/{id}" \ -u key:secret ``` POST `/v1/app-automate/maestro/{project_id}/{run_id}/cancel` ## Cancel a Maestro run Immediately cancels a queued or running Maestro run. The run is marked CANCELLED so its workers stop requesting devices and stop retrying, every flow that has not finished yet is closed out as cancelled, any device session the run already acquired is released, and any request still queued on the grid is dropped. Flows that already completed keep their results and reports. Cancelling is idempotent only in the sense that it is safe to call once: a run that has already reached a terminal state (DONE, FAILED or CANCELLED) returns 409. ### Arguments - **`project_id` integer required:** Numeric Maestro project ID the run belongs to. - **`run_id` integer required:** Numeric Maestro run ID to cancel. ### Response fields - **`success` boolean:** Whether the cancel was applied. - **`run` object:** The run in its post-cancel state. - **`cancelled_flow_count` integer:** How many flows were still queued or running and got closed out. 0 means the run held no unfinished flows (its device session, if any, is still released). POST `/v1/app-automate/maestro/{project_id}/{run_id}/cancel` [cURL](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/app-automate/maestro/{project_id}/{run_id}/cancel" \ -u key:secret ``` GET `/v1/app-automate/maestro/{project_id}/{run_id}/flow/{flow_id}` ## Get a Maestro flow result Returns the per-step results, logs, and screenshots for one flow inside a Maestro run. ### Arguments - **`project_id` integer required:** — - **`run_id` integer required:** — - **`flow_id` integer required:** — ### Response fields - **`id` integer:** Unique numeric flow-attempt ID. - **`maestro_flow_id` integer:** The flow this attempt ran. - **`status` string:** WAITING, READY, DONE, FAILED or CANCELLED. - **`name` string:** Flow name(s) covered by this attempt. - **`requested_at` timestamp:** — - **`test_case_id` integer:** Test session backing this attempt. Only present once one has been allocated. - **`completed_at` timestamp:** Only present once the attempt has finished. - **`report` string:** JUnit XML for this attempt. Only present once a report exists. - **`shard_index` integer:** Shard this attempt covered. Only present on a sharded run. - **`success` integer:** 1 when the flow passed. Only present on a terminal status. - **`error_messages` array of string:** Failure detail. Only present on a terminal status. - **`test` object:** Session detail. Only present once a session has been allocated. - **`assets` object:** Logs, video and screenshots. Only present once asset processing has finished. - **`assets_synced` boolean:** Whether this attempt's assets have finished processing. GET `/v1/app-automate/maestro/{project_id}/{run_id}/flow/{flow_id}` [cURL](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/app-automate/maestro/{project_id}/{run_id}/flow/{flow_id}" \ -u key:secret ``` GET `/v1/app-automate/maestro/{project_id}/{run_id}/html_report` ## Maestro run HTML report Returns a hosted HTML report URL summarising the Maestro run results. ### Arguments - **`project_id` integer required:** — - **`run_id` integer required:** — ### Response fields - **`html_report` string:** Signed URL to the rendered HTML report. GET `/v1/app-automate/maestro/{project_id}/{run_id}/html_report` [cURL](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/app-automate/maestro/{project_id}/{run_id}/html_report" \ -u key:secret ``` GET `/v1/app-automate/maestro/{project_id}/{run_id}/html_report_detailed` ## Maestro run detailed HTML report Returns a detailed per-step HTML report URL for the Maestro run, including screenshots and logs. ### Arguments - **`project_id` integer required:** — - **`run_id` integer required:** — ### Response fields - **`html_report_detailed` string:** Signed URL to the per-step HTML report. GET `/v1/app-automate/maestro/{project_id}/{run_id}/html_report_detailed` [cURL](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/app-automate/maestro/{project_id}/{run_id}/html_report_detailed" \ -u key:secret ``` GET `/v1/app-automate/maestro/{project_id}/{run_id}/junit_report` ## Maestro run JUnit report Returns a JUnit-format XML report for a completed Maestro run. Suitable for ingestion by CI systems (Jenkins, GitHub Actions, etc.). ### Arguments - **`project_id` integer required:** — - **`run_id` integer required:** — ### Response fields - **`junit_report` string:** JUnit XML for the run, as a string. GET `/v1/app-automate/maestro/{project_id}/{run_id}/junit_report` [cURL](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/app-automate/maestro/{project_id}/{run_id}/junit_report" \ -u key:secret ``` POST `/v1/app-automate/maestro/{project_id}/{run_id}/retry` ## Retry a Maestro run Re-runs an entire Maestro run. Creates a new run on the same device configuration (capabilities) and options, re-scheduling every flow. Returns the newly created run; poll it for status. ### Arguments - **`project_id` integer required:** — - **`run_id` integer required:** — POST `/v1/app-automate/maestro/{project_id}/{run_id}/retry` [cURL](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/app-automate/maestro/{project_id}/{run_id}/retry" \ -u key:secret ``` POST `/v1/app-automate/maestro/{project_id}/{run_id}/stop` ## Stop a Maestro run Cancels a running Maestro run before it completes. Already-completed flows are preserved; in-flight flows are marked cancelled. Alias of the cancel endpoint, kept because testingbotctl calls it from its Ctrl-C handler; unlike cancel it always answers 200, so stopping an already-finished run is a no-op rather than a 409. ### Arguments - **`project_id` integer required:** — - **`run_id` integer required:** — POST `/v1/app-automate/maestro/{project_id}/{run_id}/stop` [cURL](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/app-automate/maestro/{project_id}/{run_id}/stop" \ -u key:secret ``` POST `/v1/app-automate/maestro/{project_id}/{run_id}/{flow_id}/retry` ## Retry a Maestro flow Re-runs a single flow from a Maestro run. Creates a new flow attempt inside the same run and schedules it. Returns the newly created flow record; poll the run for its status. ### Arguments - **`project_id` integer required:** — - **`run_id` integer required:** — - **`flow_id` integer required:** — POST `/v1/app-automate/maestro/{project_id}/{run_id}/{flow_id}/retry` [cURL](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/app-automate/maestro/{project_id}/{run_id}/{flow_id}/retry" \ -u key:secret ``` ## App Automate: XCUITest POST `/v1/app-automate/xcuitest/app` ## Upload an iOS app for XCUITest Uploads the production iOS `.ipa` you want to test under XCUITest. POST `/v1/app-automate/xcuitest/app` [cURL](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/app-automate/xcuitest/app" \ -u key:secret ``` GET `/v1/app-automate/xcuitest/{id}` ## Get an XCUITest project Returns the metadata and current state of an XCUITest project. ### Arguments - **`id` integer required:** — ### Response fields - **`runs` array of object:** Runs belonging to the project. - **`success` boolean:** True when every run in the project passed. - **`completed` boolean:** True when every run has reached a terminal state. GET `/v1/app-automate/xcuitest/{id}` [cURL](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/app-automate/xcuitest/{id}" \ -u key:secret ``` GET `/v1/app-automate/xcuitest/{id}/report` ## Get the XCUITest JUnit report Returns the JUnit-format XML report for a completed XCUITest run. ### Arguments - **`id` integer required:** — GET `/v1/app-automate/xcuitest/{id}/report` [cURL](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/app-automate/xcuitest/{id}/report" \ -u key:secret ``` POST `/v1/app-automate/xcuitest/{id}/run` ## Run an XCUITest suite Triggers an asynchronous XCUITest run across the requested device capabilities. ### Arguments - **`id` integer required:** — POST `/v1/app-automate/xcuitest/{id}/run` [cURL](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/app-automate/xcuitest/{id}/run" \ -u key:secret ``` POST `/v1/app-automate/xcuitest/{id}/tests` ## Upload XCUITest test bundle Attaches the XCUITest test bundle (.zip) to a project. Pair with `/app` to provide both halves of the test. ### Arguments - **`id` integer required:** — POST `/v1/app-automate/xcuitest/{id}/tests` [cURL](https://testingbot.com#) Request ```bash $ curl -X POST "https://api.testingbot.com/v1/app-automate/xcuitest/{id}/tests" \ -u key:secret ``` GET `/v1/app-automate/xcuitest/{project_id}/{id}` ## Get a specific XCUITest run Returns the detail record for a single XCUITest run inside a project. ### Arguments - **`project_id` integer required:** — - **`id` integer required:** — ### Response fields - **`id` integer:** Unique numeric XCUITest run ID. - **`created_at` timestamp:** — - **`status` string:** WAITING, READY, DONE or FAILED. - **`capabilities` object:** Capabilities the run was started with. - **`success` boolean:** Whether the run passed. False until a session has completed. - **`test` object:** Session detail. Only present once a session has been allocated. GET `/v1/app-automate/xcuitest/{project_id}/{id}` [cURL](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/app-automate/xcuitest/{project_id}/{id}" \ -u key:secret ``` GET `/v1/app-automate/xcuitest/{project_id}/{run_id}/html_report` ## XCUITest run HTML report Returns a self-contained HTML report for a single XCUITest run. ### Arguments - **`project_id` integer required:** — - **`run_id` integer required:** — ### Response fields - **`html_report` string:** Signed URL to the rendered HTML report. GET `/v1/app-automate/xcuitest/{project_id}/{run_id}/html_report` [cURL](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/app-automate/xcuitest/{project_id}/{run_id}/html_report" \ -u key:secret ``` GET `/v1/app-automate/xcuitest/{project_id}/{run_id}/junit_report` ## XCUITest run JUnit report Returns a JUnit-format XML report for a single completed XCUITest run. Suitable for ingestion by CI systems (Jenkins, GitHub Actions, etc.). ### Arguments - **`project_id` integer required:** — - **`run_id` integer required:** — ### Response fields - **`junit_report` string:** JUnit XML for the run, as a string. GET `/v1/app-automate/xcuitest/{project_id}/{run_id}/junit_report` [cURL](https://testingbot.com#) Request ```bash $ curl "https://api.testingbot.com/v1/app-automate/xcuitest/{project_id}/{run_id}/junit_report" \ -u key:secret ``` [Previous Webhooks](https://testingbot.com/support/api/webhooks) --- URL: https://testingbot.com/support/k6-browser-testing # k6 Browser Testing k6 is an open-source tool designed to do easy load testing against websites. With the [k6 browser module](https://grafana.com/docs/k6/latest/using-k6-browser/), you can connect to a CDP-compatible browser grid such as TestingBot. Simply configure your k6 test to use the TestingBot grid, and instantly get access to a high capacity cloud of browsers. With this integration, you can run tests in parallel, targeting different browsers, versions and operating systems. Each test comes with logs, screenshots and a video. ## Setup To get started, please make sure you [install k6](https://k6.io/docs/get-started/installation/). ### macOS ```bash brew install k6 ``` ### Windows ```bash winget install k6 --source winget ``` ### Linux ```bash sudo gpg -k sudo gpg --no-default-keyring --keyring /usr/share/keyrings/k6-archive-keyring.gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C5AD17C747E3415A3642D57D77C6C491D6AC1D69 echo "deb [signed-by=/usr/share/keyrings/k6-archive-keyring.gpg] https://dl.k6.io/deb stable main" | sudo tee /etc/apt/sources.list.d/k6.list sudo apt-get update sudo apt-get install k6 ``` ## Example Next, let's run a sample browser test using k6 on the TestingBot browser grid. Please see the example below. To connect to the TestingBot CDP grid, you will need to specify a `K6_BROWSER_WS_URL` environment variable. This needs to contain the URL to the TestingBot grid, together with the credentials and (url-encoded) capabilities. ```bash K6_BROWSER_WS_URL="wss://cloud.testingbot.com/?key=API_KEY&secret=API_SECRET&capabilities=%7B%22browserName%22%3A%22chrome%22%2C%22browserVersion%22%3A%22latest%22%7D" ``` To run the example below, you can use the following command in your terminal: ```bash K6_BROWSER_WS_URL="wss://cloud.testingbot.com/?key=API_KEY&secret=API_SECRET&capabilities=%7B%22browserName%22%3A%22chrome%22%2C%22browserVersion%22%3A%22latest%22%7D" K6_BROWSER_ENABLED=true k6 run k6_sample.js ``` **k6\_sample.js** ```javascript import { browser } from 'k6/browser'; import { check } from 'k6'; export const options = { scenarios: { ui: { executor: 'shared-iterations', options: { browser: { type: 'chromium', }, }, }, }, thresholds: { checks: ['rate==1.0'], }, }; export default async function () { const context = await browser.newContext(); const page = await context.newPage(); try { await page.goto('https://test.k6.io/my_messages.php'); await page.locator('input[name="login"]').type('admin'); await page.locator('input[name="password"]').type('123'); const submitButton = page.locator('input[type="submit"]'); await Promise.all([page.waitForNavigation(), submitButton.click()]); const header = await page.locator('h2').textContent(); check(page, { header: header == 'Welcome, admin!', }); } finally { await page.close(); } } ``` ### K6\_BROWSER\_WS\_URL To construct the `K6_BROWSER_WS_URL`, you'll need to urlencode the capabilities: ```javascript const capabilities = { browserName: 'chrome', browserVersion: 'latest', platformName: 'VENTURA' } const url = `wss://cloud.testingbot.com/?key=API_KEY&secret=API_SECRET&capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` ``` ## Test Results You can find the results of your k6 browser test in your terminal. The TestingBot member area will contain a list of all k6 tests, together with logs, screenshots and a video of each test. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate Documentation # Mobile App Test Automation TestingBot runs automated tests for native iOS, Android and cross-platform apps on real iPhones, iPads and Android phones and tablets in our EU datacenter, plus iOS simulators and Android emulators for fast iteration. Bring your existing Appium, XCUITest, Espresso, Maestro, Mobilewright or Electron tests and point them at the TestingBot grid. ## Pick a mobile test framework TestingBot supports the major mobile and cross-platform test frameworks. Each card links to a complete setup guide with code examples in every supported language. Not sure which framework to pick? See the [framework comparison](https://testingbot.com#compare) below. ![Appium](https://testingbot.com/assets/support/appium-4b35a8bb56bae236bd84c539cee9b8bbb489a55ddf48e32d059c86685d230354.svg) ### [Appium](https://testingbot.com/support/app-automate/appium) W3C WebDriver framework for cross-platform iOS and Android app automation. - [Java](https://testingbot.com/support/app-automate/appium/java) - [Python](https://testingbot.com/support/app-automate/appium/python) - [Node.js](https://testingbot.com/support/app-automate/appium/nodejs) - [Ruby](https://testingbot.com/support/app-automate/appium/ruby) - [C#](https://testingbot.com/support/app-automate/appium/csharp) - [PHP](https://testingbot.com/support/app-automate/appium/php) - [Flutter](https://testingbot.com/support/app-automate/appium/flutter) ![XCUITest](https://testingbot.com/assets/integrations/xcuitest-4a935f2368661d89d3559b3c5af3377236029478ae76bb43253f3c3bee712096.svg) ### [XCUITest](https://testingbot.com/support/app-automate/xcuitest) Apple's native iOS UI testing framework, written in Swift or Objective-C. - [XCUITest guide](https://testingbot.com/support/app-automate/xcuitest) ![Espresso](https://testingbot.com/assets/integrations/espresso-a8ff30da0c385f89f4e694bb29fc4779ad132e8e3f4a777a38e608e0492ff610.svg) ### [Espresso](https://testingbot.com/support/app-automate/espresso) Google's native Android UI testing framework, written in Java or Kotlin. - [Espresso guide](https://testingbot.com/support/app-automate/espresso) ![Maestro](https://testingbot.com/assets/integrations/maestro-9d782f30f13fb4efaf9928b704c7a787eb14cb9e5b20c650137bb65892334b8b.svg) ### [Maestro](https://testingbot.com/support/app-automate/maestro) YAML-based declarative test runner for iOS and Android. - [Maestro guide](https://testingbot.com/support/app-automate/maestro) ![Mobilewright](https://testingbot.com/assets/integrations/mobilewright-fdc12119ed8fb1a08f7d6116eff6fa37f5c4616acccc2b22eca478a707fd3870.png) ### [Mobilewright](https://testingbot.com/support/app-automate/mobilewright) Playwright-style mobile tests in TypeScript for iOS and Android. - [Mobilewright guide](https://testingbot.com/support/app-automate/mobilewright) ![Electron](https://testingbot.com/assets/environments/svg/electron-91b2641c5cea963ea9c261f274ae480fc4fbc689889a058158c9ebbcd559b622.svg) ### [Electron](https://testingbot.com/support/app-automate/electron) Test cross-platform Electron desktop apps built with web technologies. - [Electron guide](https://testingbot.com/support/app-automate/electron) ### [Smart TV](https://testingbot.com/support/app-automate/smart-tv) Automate Smart TV and OTT applications on real connected TV hardware. - [Smart TV guide](https://testingbot.com/support/app-automate/smart-tv) ## Quick start: run your first mobile app test Four steps to point an existing Appium, XCUITest, Espresso or Maestro test at the TestingBot cloud: 1. **Sign up and grab your API key.** [Create a free account](https://testingbot.com/users/sign_up). Your API key and secret are in the dashboard under Account, then Keys and Authentication. 2. **Upload your app binary to TestingBot Storage.** Upload your `.ipa` (iOS), `.apk` (Android) or Maestro flow ZIP to [TestingBot Storage](https://testingbot.com/support/api/storage#upload). The response returns a `tb://` URL you reference in test capabilities. 3. **Point your framework at the TestingBot endpoint.** Connect to `https://hub.testingbot.com/wd/hub`. Pass your key and secret via `tb:options`, set `platformName`, `deviceName`, `platformVersion` and `app` (the `tb://` URL). 4. **Run the test and view results.** TestingBot streams a live video, screenshots, device logs and framework command logs into the dashboard while the test runs and after it finishes. ## Appium vs XCUITest vs Espresso vs Maestro All four frameworks run on TestingBot. Pick the one that fits your codebase and team: | | Appium | XCUITest | Espresso | Maestro | | Platforms | iOS and Android | iOS only | Android only | iOS and Android | | Languages | Java, Python, Ruby, JavaScript, C#, PHP, Flutter | Swift, Objective-C | Java, Kotlin | YAML (declarative) | | Protocol | W3C WebDriver | Apple XCTest runner | Google AndroidJUnitRunner | Maestro CLI | | Test speed | Slower (extra protocol layer) | Fast (native runner) | Fast (native runner) | Fast (direct UI driver) | | Best for | One codebase across iOS and Android, language flexibility | iOS teams already using XCTest in Xcode | Android teams already using AndroidJUnit in Studio | QA teams that want readable, low-code flows | ## Frequently asked questions What is mobile app test automation? Mobile app test automation runs UI tests against native iOS and Android applications, exercising the app exactly like a user would. TestingBot runs these tests on real iPhones, iPads, Android phones and tablets in our EU datacenter, plus iOS simulators and Android emulators when you want faster iteration. See the [supported device list](https://testingbot.com/support/app-automate/devices) for current models. Which mobile test frameworks does TestingBot support? TestingBot supports Appium for cross-platform iOS and Android, Apple's XCUITest for iOS, Google's Espresso for Android, Maestro for declarative cross-platform flows, Mobilewright for Playwright-style tests in TypeScript, and Electron for cross-platform desktop apps. See the dedicated guides for [Appium](https://testingbot.com/support/app-automate/appium), [XCUITest](https://testingbot.com/support/app-automate/xcuitest), [Espresso](https://testingbot.com/support/app-automate/espresso), [Maestro](https://testingbot.com/support/app-automate/maestro), [Mobilewright](https://testingbot.com/support/app-automate/mobilewright) and [Electron](https://testingbot.com/support/app-automate/electron). Can I run tests on real iPhones and Android devices, not simulators? Yes. TestingBot runs automated tests on physical iPhones, iPads, and Android phones and tablets housed in our EU datacenter, alongside iOS simulators and Android emulators. Use `realDevice: true` in your capabilities to target real hardware. See [real device testing](https://testingbot.com/real-device-testing) for the full real-hardware overview. How do I upload my app binary to TestingBot? Upload your `.ipa`, `.apk` or Maestro flow ZIP to [TestingBot Storage](https://testingbot.com/support/api/storage#upload) with your API key and secret. The response returns a `tb://` URL that you reference in your test capabilities. Apps are retained for 60 days and can be deleted via the REST API at any time. Can I run mobile app tests in parallel? Yes. Each plan includes a number of concurrent device sessions. Most test runners (Appium, XCUITest, Espresso, Maestro) launch parallel sessions natively, and each session runs on a separate, isolated device. See [our pricing plans](https://testingbot.com/pricing) for the device-concurrency limits. What is the difference between Appium, XCUITest, Espresso and Maestro? Appium is a cross-platform W3C WebDriver implementation for iOS and Android, similar to Selenium. XCUITest is Apple's native iOS UI testing framework. Espresso is Google's native Android UI testing framework. Maestro is a YAML-based declarative test runner for both platforms. Appium gives the broadest reuse for teams that need one codebase across iOS and Android. XCUITest and Espresso are tightly coupled to their respective platforms and typically run faster. Maestro is the simplest to write and read. See the [framework comparison](https://testingbot.com#frameworks) on this page for details. Can I test my app on a real device behind a firewall or staging environment? Yes. Use the [TestingBot Tunnel](https://testingbot.com/support/tunnel) to expose internal services to the device under test. Start the tunnel with an identifier, then pass tunnelIdentifier in your tb:options capabilities. The tunnel encrypts traffic between your network and the cloud device. How are mobile app tests billed on TestingBot? Mobile app tests are billed by the minute (the time the device was allocated to your test) and capped by your plan's concurrency limit. Simulator, emulator and real-device sessions all consume the same minutes at the same rate. See [our pricing plans](https://testingbot.com/pricing) for current rates. ## Next steps - [ **Supported devices** Real iPhones, iPads and Android devices on the grid](https://testingbot.com/support/app-automate/devices) - [ **Appium guide** Cross-platform iOS and Android automation](https://testingbot.com/support/app-automate/appium) - [ **XCUITest guide** Native iOS UI testing](https://testingbot.com/support/app-automate/xcuitest) - [ **Espresso guide** Native Android UI testing](https://testingbot.com/support/app-automate/espresso) - [ **Maestro guide** YAML-based declarative flows](https://testingbot.com/support/app-automate/maestro) - [ **Mobilewright guide** Playwright-style tests in TypeScript](https://testingbot.com/support/app-automate/mobilewright) - [ **Real device testing** How the real-hardware cloud works](https://testingbot.com/real-device-testing) - [ **TestingBot Tunnel** Test against internal or staging APIs](https://testingbot.com/support/tunnel) - [ **Pricing** Real-device plans and concurrency](https://testingbot.com/pricing) ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/devices - [Desktop Browsers & Emulators](https://testingbot.com/support/web-automate/browsers) - [Physical Mobile Devices](https://testingbot.com#) # Physical Mobile Devices Run Appium, Espresso and XCUITest automation on our farm of real Android and iOS devices. Pick a device to see the exact capabilities for your test framework. Platform - All Platforms - Apple iOS - Android Android Version - All Versions - Android 17.0 - Android 16.0 - Android 15.0 - Android 14.0 - Android 13.0 - Android 12.0 - Android 11.0 - Android 10.0 - Android 9.0 - Android 7.1 - Android 6.0 iOS Version - All Versions - iOS 26.6 - iOS 26.5 - iOS 26.4 - iOS 26.2 - iOS 26.0 - iOS 18.7 - iOS 18.5 - iOS 18.0 - iOS 17.6 - iOS 17.5 - iOS 16.7 - iOS 16.5 - iOS 15.8 - iOS 15.0 - iOS 14.8 - iOS 14.7 - iOS 14.5 - iOS 14.0 - iOS 13.5 - iOS 12.2 Resolution - All Resolutions - 1440x3440 - 1440x3200 - 1320x2868 - 1260x2736 - 1206x2622 - 1179x2556 - 1170x2532 - 1080x2424 - 1080x2400 - 1080x2340 - 1080x2220 - 1200x1920 - 1080x1920 - 828x1792 - 720x1600 - 750x1334 - 810x1080 - 768x1024 - 758x1011 - ![Test online on Galaxy S8](https://testingbot.com/assets/devices/160h/1-1f8a8f10826bfb4e6109e741abfaa6de5cc97901aa7626b6e396e65d14b93cc8.png) Galaxy S8 Android 9.0 1080x2220 - **CPU::** Octa-core (4x2.3 GHz Mongoose M2 & 4x1.7 GHz Cortex-A53) - **Model::** SM-G950U1 Capabilities - ![Test online on iPhone 6s](https://testingbot.com/assets/devices/160h/2-67e8216b79cb1c4fc2d07b796042245e830042550a7adf3ebb1d8287da9715ad.png) iPhone 6s iOS 15.8 750x1334 - **CPU::** Apple A9 - **Model::** MKQN2ZD/A Capabilities - ![Test online on iPad (6th generation)](https://testingbot.com/assets/devices/160h/3-db10ee5ecdf299dd3932e8938c802d0e57265406a5f3bc522841b72b84cb0d96.png) iPad (6th generation) iOS 13.5 768x1024 - **CPU::** Apple A10 - **Model::** MR7G2NF/A Capabilities - ![Test online on iPhone XR](https://testingbot.com/assets/devices/160h/5-7ca9c0a7d4fe6708a2c08d25bd23663169243c9f90807be323ab428590491cf9.png) iPhone XR iOS 18.7 828x1792 - **CPU::** Apple A12 Bionic - **Model::** MRY42B/A Capabilities - ![Test online on Pixel](https://testingbot.com/assets/devices/160h/7-23de2fe6c4a4119d402d125ee669fa908692350c7da36139b4702f5b184cfe3d.png) Pixel Android 7.1 1080x1920 - **CPU::** ARM - Quad-core - 1593Mhz - **Model::** PQ1A.190105.004 Capabilities - ![Test online on Galaxy S10](https://testingbot.com/assets/devices/160h/8-c140ac807c625801f548a906e01355158b71c361fdc86ddd9a645ad935827c05.png) Galaxy S10 Android 10.0 1440x3440 - **CPU::** Octa-core - **Model::** SM-G973F Capabilities - ![Test online on Galaxy Tab S11](https://testingbot.com/assets/devices/160h/9-1db1e22ab4516862797cca7693079084ea3e3582f911d7b51971754d83974752.png) Galaxy Tab S11 Android 16.0 1200x1920 - **CPU::** ARM - Quad-core - 1200Mhz - **Model::** SM-T510 Capabilities - ![Test online on iPhone 8](https://testingbot.com/assets/devices/160h/10-5d4c0cdcd2238d10c36916a95e87e7541d4ace0a45436637f51f494cf4308ede.png) iPhone 8 iOS 16.7 750x1334 - **CPU::** ARM - Dual-core - 1840Mhz - **Model::** MQ6H2 Capabilities - ![Test online on iPhone 11](https://testingbot.com/assets/devices/160h/11-3ddb6dcb39e2d4e307eba6b52ba4bad5839054feb89abcd74cae48235edab46d.png) iPhone 11 iOS 14.0 828x1792 - **CPU::** - **Model::** MWLT2 Capabilities - ![Test online on Galaxy S20](https://testingbot.com/assets/devices/160h/12-3d5eea05f5802ad49748405da14a5531d07a24e3b5a9fe52c8ca2b29c63a6774.png) Galaxy S20 Android 10.0 1440x3200 - **CPU::** ARM - Quad-core - 1593Mhz - **Model::** E Capabilities - ![Test online on iPhone 12](https://testingbot.com/assets/devices/160h/14-feb85685f170057dbc2d7c6295001ac6d8bc52d0c0e0ba03053d779f4f73ad9c.png) iPhone 12 iOS 14.5 1170x2532 - **CPU::** Apple A14 Bionic - **Model::** MWLT2 Capabilities - ![Test online on Galaxy S21](https://testingbot.com/assets/devices/160h/15-540f427f967d455d2b8b95b70aada460fce369c428998422462dfd56d56013f6.png) Galaxy S21 Android 11.0 1080x2400 - **CPU::** ARM - Quad-core - 1593Mhz - **Model::** E Capabilities - ![Test online on Mi 10](https://testingbot.com/assets/devices/160h/16-7627417896e3822627bc0d60fc33c605b7db401a8c58527834407506faf43fab.png) Mi 10 Android 12.0 1080x2400 - **CPU::** ARM - Single-core - 1690Mhz - **Model::** M2007J3SY Capabilities - ![Test online on Galaxy A12](https://testingbot.com/assets/devices/160h/17-a6829514cbda5863012c38310b62729e9c30f4e594ef084bc5b374cd18d53fe0.png) Galaxy A12 Android 11.0 720x1600 - **CPU::** - **Model::** SM-A125F Capabilities - ![Test online on iPad (8th generation)](https://testingbot.com/assets/devices/160h/18-efdf40f2da9a603ff90fb75e8cc846ac7e406c625a2bb4322680e71fea50a8de.png) iPad (8th generation) iOS 15.0 758x1011 - **CPU::** ARM - Quad-core - 2340MHz - **Model::** MR7G2NF/A Capabilities - ![Test online on iPhone XR](https://testingbot.com/assets/devices/160h/19-7ca9c0a7d4fe6708a2c08d25bd23663169243c9f90807be323ab428590491cf9.png) iPhone XR iOS 14.8 828x1792 - **CPU::** ARM - Single-core - 2390Mhz - **Model::** MRY42B/A Capabilities - ![Test online on iPhone 13](https://testingbot.com/assets/devices/160h/22-69483440c27820ca3dbcc051d3823215e595f61aa21b768460791389612bf12b.png) iPhone 13 iOS 18.5 1170x2532 - **CPU::** ARM - **Model::** Capabilities - ![Test online on Galaxy S23](https://testingbot.com/assets/devices/160h/23-c2df9301b8ca06b86395ca220b717010b21a0a0fc552b1e3be83112ac18b0988.png) Galaxy S23 Android 13.0 1080x2340 - **CPU::** ARM - **Model::** Capabilities - ![Test online on iPhone 14](https://testingbot.com/assets/devices/160h/24-334a9172a1cd4d0ebcdafd77a15bae6eedcbccffd8470ac5735f50570ba74620.png) iPhone 14 iOS 17.6 1170x2532 - **CPU::** ARM - **Model::** Capabilities - ![Test online on iPhone 11](https://testingbot.com/assets/devices/160h/25-3ddb6dcb39e2d4e307eba6b52ba4bad5839054feb89abcd74cae48235edab46d.png) iPhone 11 iOS 14.8 828x1792 - **CPU::** - **Model::** MWLT2 Capabilities - ![Test online on iPad (8th generation)](https://testingbot.com/assets/devices/160h/26-3dfdc59679360eafc8969822e41f13f9fca3020e011ec31e38e7eed0617fd042.png) iPad (8th generation) iOS 14.7 810x1080 - **CPU::** ARM - Quad-core - 2340MHz - **Model::** MR7G2NF/A Capabilities - ![Test online on iPhone 15](https://testingbot.com/assets/devices/160h/27-fa43101c0c582cc4892313c96ddcdf88ccae7c4ffff425cd20989099345ea1d0.png) iPhone 15 iOS 18.0 1179x2556 - **CPU::** ARM - **Model::** Capabilities - ![Test online on Pixel 8](https://testingbot.com/assets/devices/160h/28-98ab89fcf1461b958032b26327abb47a842592cbc341b0522de3c1b21901763c.png) Pixel 8 Android 14.0 1080x2400 - **CPU::** Google Tensor G3 (4 nm) - **Model::** GKWS6 Capabilities - ![Test online on Galaxy S24](https://testingbot.com/assets/devices/160h/29-721713244928fc0799e7e9d60e3e260e811243489e2d572d6009fb7c8ed9019d.png) Galaxy S24 Android 14.0 1080x2340 - **CPU::** ARM - **Model::** Capabilities - ![Test online on Galaxy A55](https://testingbot.com/assets/devices/160h/30-4e523289f8fe0edae798e39d71f3c0cf7930cbdc4807771da84e201ec7dd77f0.png) Galaxy A55 Android 14.0 1080x2340 - **CPU::** Exynos 1480 - **Model::** SM-A556ELBHINS Capabilities - ![Test online on iPhone SE 2022](https://testingbot.com/assets/devices/160h/31-90fa36ca0531263e241615775883dab78816e6c91e74d66e7603e5b8b7a16928.png) iPhone SE 2022 iOS 17.5 750x1334 - **CPU::** A15 Bionic Chip - **Model::** A2783 Capabilities - ![Test online on Redmi Note 13](https://testingbot.com/assets/devices/160h/32-2a98c9f3e70285cd2c639462e106b739d893a41ed98e63994e4c24c327ab5e77.png) Redmi Note 13 Android 14.0 1080x2400 - **CPU::** Octa-core (2x2.4 GHz Cortex-A76) - **Model::** 23124RA7EO Capabilities - ![Test online on iPhone 16](https://testingbot.com/assets/devices/160h/33-3fe3f72dccbdf4320c88d81c18a9ae9c28dcbd1023e85b71db659ddd1c41752e.png) iPhone 16 iOS 26.5 1179x2556 - **CPU::** Apple A18 - **Model::** A3287 Capabilities - ![Test online on Galaxy S25](https://testingbot.com/assets/devices/160h/34-66fda97a65984960824cbfcb57ad4458c0e9a5d7904982bcb5506776013338c5.png) Galaxy S25 Android 15.0 1080x2340 - **CPU::** ARM - **Model::** Capabilities - ![Test online on Pixel 9](https://testingbot.com/assets/devices/160h/35-3fa3958dd941e15ecfde215bd9d4014952803aef46a16245e04187e824ec59ce.png) Pixel 9 Android 16.0 1080x2424 - **CPU::** Google Tensor G4 - **Model::** GUR25 Capabilities - ![Test online on iPhone 17](https://testingbot.com/assets/devices/160h/36-9527070deeb6aceeb90648e8df41f90d7443eb08b7b41625451dc2f861af845f.png) iPhone 17 iOS 26.0 1179x2556 - **CPU::** Apple A19 - **Model::** Capabilities - ![Test online on Pixel 10](https://testingbot.com/assets/devices/160h/37-d53c19554e3e5731d4238e1c72be755b39eeafafc391bd0da2e92fc97659ee59.png) Pixel 10 Android 16.0 1080x2424 - **CPU::** Google Tensor G5 - **Model::** GUR25 Capabilities - ![Test online on iPhone Air](https://testingbot.com/assets/devices/160h/38-d22d9cb8c40d79cfe188f9ab5691b4ef5db63551cd2ae71ba5ecc5cfd59e5398.png) iPhone Air iOS 26.2 1260x2736 - **CPU::** Apple A19 Pro - **Model::** Capabilities - ![Test online on Huawei P40](https://testingbot.com/assets/devices/160h/39-05227623a4f02381c190b5e7af808c32db0708e365cc6184f1c86f0008cf12e6.png) Huawei P40 Android 12.0 1080x2340 - **CPU::** Cortex-A76 - **Model::** ANA-AN00 Capabilities - ![Test online on Galaxy S26](https://testingbot.com/assets/devices/160h/40-f50120b3213ff29f42bd94653d5e980e86cd36461196d636cf474e503401a770.png) Galaxy S26 Android 16.0 1080x2340 - **CPU::** ARM - **Model::** Capabilities - ![Test online on iPad (9th generation)](https://testingbot.com/assets/devices/160h/41-3dfdc59679360eafc8969822e41f13f9fca3020e011ec31e38e7eed0617fd042.png) iPad (9th generation) iOS 26.6 810x1080 - **CPU::** ARM - Quad-core - 2340MHz - **Model::** Capabilities - ![Test online on iPhone 16 Pro](https://testingbot.com/assets/devices/160h/42-9706067a61606da62bb3cbb699085e76db69948c6b9afdee47fc7dcff3449737.png) iPhone 16 Pro iOS 26.4 1206x2622 - **CPU::** Apple A18 Pro - **Model::** D93AP Capabilities - eSIM ![Test online on iPhone 17 Pro Max](https://testingbot.com/assets/devices/160h/43-67eb7da109303355ff55e0477cbe56c9bae21c4a027a6aa66a01ea82a6a6b499.png) iPhone 17 Pro Max iOS 26.6 1320x2868 - **CPU::** Apple A19 Pro - **Model::** Capabilities - ![Test online on iPhone 17 Pro](https://testingbot.com/assets/devices/160h/44-853d3b10f125a9e69f1dba45db82a5a9a87a19c35c3a07b980a72926bb850eb0.png) iPhone 17 Pro iOS 26.5 1206x2622 - **CPU::** Apple A19 Pro - **Model::** Capabilities - ![Test online on Pixel 11](https://testingbot.com/assets/devices/160h/45-8d82669860de455557faf9108a0c4d41e21d3bef2d5dafba9dc7616b0294a2ea.png) Pixel 11 Android 17.0 1080x2424 - **CPU::** Google Tensor G6 - **Model::** GUJ0N Capabilities - ![Test online on Pixel 8](https://testingbot.com/assets/devices/160h/46-98ab89fcf1461b958032b26327abb47a842592cbc341b0522de3c1b21901763c.png) Pixel 8 Android 17.0 1080x2400 - **CPU::** Google Tensor G3 (4 nm) - **Model::** GKWS6 Capabilities No devices match your filters Try removing a filter or clearing your search. ## Capabilities Use these Appium capabilities to run your test on this device. --- URL: https://testingbot.com/support/app-automate/electron # Electron Testing [Electron](https://electronjs.org/) is a framework that allows you to create cross-platform native applications by using Javascript code. Electron will embed a Chromium engine in a native app format, allowing web developers to create desktop applications by writing HTML and Javascript code. TestingBot supports running automated WebDriver tests against Electron-based apps on these platforms. Prefer Playwright? See [Electron testing with Playwright](https://testingbot.com/support/app-automate/electron/playwright). - Windows 11 - Windows 10 - macOS Tahoe - macOS Sequoia - macOS Sonoma - macOS Ventura - macOS Monterey All Electron versions from version 16 to the most recent version are supported. TestingBot has created a [demo Electron app](https://github.com/testingbot/testingbot-electron-demo-app), which acts as a simple calculator, to showcase automated Electron app testing. ## Run your first Electron test ### App management To test an Electron app, you'll first need to upload a zip file of your Electron application to a URL where TestingBot can download it from. You can choose to upload this yourself, for example to AWS S3 or use [TestingBot Storage](https://testingbot.com/support/api/storage#upload) to upload your `.zip` file. Note that files uploaded to TestingBot Storage are automatically pruned after 60 days. [cURL](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) ```bash $ curl -X POST "https://api.testingbot.com/v1/storage" \ -u key:secret -F "file=@/path/to/app/file/electron-app.zip" ``` ```ruby require 'testingbot' api = TestingBot::Api.new("key", "secret") api.upload_local_file('/path/to/electron-app.zip') ``` ```java TestingbotREST restApi = new TestingbotREST("key", "secret"); TestingbotStorageUploadResponse uploadResponse = restApi.uploadToStorage(new File("/path/to/electron-app.zip")); ``` ```php $api = new TestingBot\TestingBotAPI("key", "secret"); $api->uploadLocalFileToStorage("/path/to/electron-app.zip"); ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient("key", "secret") tb.storage.upload_local_file("/path/to/electron-app.zip") ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); api.uploadFile('/path/to/electron-app.zip', function(error, response) { // response.app_url }); ``` TestingBot Storage will return a `app_url` which you can use in the `tb:app` capability of your test. This is how the capabilities in your test should currently look like: [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { "browserName" => "electron", "browserVersion" => "42", "tb:app" => "tb://the-app-url-you-received" # or https://s3.amazonaws.com/... } ``` ```java ChromeOptions options = new ChromeOptions(); options.setCapability("browserName", "electron"); options.setCapability("browserVersion", "42"); options.setCapability("tb:app", "tb://the-app-url-you-received"); Map tbOptions = new HashMap<>(); tbOptions.put("key", "api_key"); tbOptions.put("secret", "api_secret"); options.setCapability("tb:options", tbOptions); ``` ```python from selenium import webdriver from selenium.webdriver.chrome.options import Options options = Options() options.set_capability("browserName", "electron") options.set_capability("browserVersion", "42") options.set_capability("tb:app", "tb://the-app-url-you-received") options.set_capability("tb:options", { "name": "Electron Sample" }) driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'electron', "browserVersion": "42", "tb:options": { "key": "api_key", "secret": "api_secret" }, 'tb:app': 'tb://the-app-url-you-received' // or https://s3.amazonaws.com/... }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions(); chromeOptions.BrowserName = "electron"; chromeOptions.BrowserVersion = "42"; chromeOptions.PlatformName = "Windows 10"; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["name"] = TestContext.CurrentContext.Test.Name }; // Use AddAdditionalOption (Selenium 4) so these vendor caps land at the top level, // not nested under goog:chromeOptions like AddAdditionalChromeOption would. chromeOptions.AddAdditionalOption("tb:options", tbOptions); chromeOptions.AddAdditionalOption("tb:app", "tb://the-app-url-you-received"); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions); ``` ### Binary location Next to the required `app` capability, you will also need to specify a `tb:binary_location` capability. This is the path to the actual Electron app, so that TestingBot knows which file in the `zip` file is the Electron app. For example, if your uploaded zip file is structured like this: ```yaml TestingBot-Electron-App.zip [testingbot-electron-demo-app.app] [Contents] [MacOS] -- testingbot-electron-demo-app ``` The `binary_location` capability in this case would be: ```bash tb:binary_location = "testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app" ``` Now the capabilities in your test should look like the example below. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby require 'selenium-webdriver' options = Selenium::WebDriver::Options.chrome options.browser_name = "electron" options.platform_name = "TAHOE" options.browser_version = "31" options.add_option("tb:binary_location", "testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app") options.add_option("tb:app", "https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-darwin-arm64-1.0.0.zip") http_client = Selenium::WebDriver::Remote::Http::Default.new http_client.open_timeout = 200 http_client.read_timeout = 200 driver = Selenium::WebDriver.for( :remote, url: "https://key:secret@hub.testingbot.com/wd/hub", options: options, http_client: http_client) # Generate two random numbers between 0 and 9 num1 = rand(0..9) num2 = rand(0..9) # Find elements on the page first_number = driver.find_element(id: "btn-#{num1}") second_number = driver.find_element(id: "btn-#{num2}") plus = driver.find_element(id: 'btn-plus') equal = driver.find_element(id: 'btn-equal') # Perform operations first_number.click plus.click second_number.click equal.click # Get the result result_element = driver.find_element(id: 'calc-display') result_text = result_element.text driver.quit raise "Test failed" unless (num1 + num2) == result_text.to_i ``` ```java import org.openqa.selenium.By; import org.openqa.selenium.WebDriver; import org.openqa.selenium.WebElement; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URL; import java.util.Random; public class SeleniumTest { public static void main(String[] args) { try { ChromeOptions options = new ChromeOptions(); options.setCapability("browserName", "electron"); options.setCapability("platformName", "TAHOE"); options.setCapability("browserVersion", "31"); options.setCapability("tb:binary_location", "testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app"); options.setCapability("tb:app", "https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-darwin-arm64-1.0.0.zip"); WebDriver driver = new RemoteWebDriver( new URL("https://key:secret@hub.testingbot.com/wd/hub"), options ); // Generate two random numbers between 0 and 9 Random random = new Random(); int num1 = random.nextInt(10); int num2 = random.nextInt(10); WebElement firstNumber = driver.findElement(By.id("btn-" + num1)); WebElement secondNumber = driver.findElement(By.id("btn-" + num2)); WebElement plus = driver.findElement(By.id("btn-plus")); WebElement equal = driver.findElement(By.id("btn-equal")); firstNumber.click(); plus.click(); secondNumber.click(); equal.click(); WebElement resultElement = driver.findElement(By.id("calc-display")); String resultText = resultElement.getText(); // Verify the result if ((num1 + num2) != Integer.parseInt(resultText)) { throw new AssertionError("Test failed"); } // Quit the driver driver.quit(); } catch (Exception e) { e.printStackTrace(); } } } ``` ```python from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.chrome.options import Options import random # Set capabilities options = Options() options.set_capability("browserName", "electron") options.set_capability("platformName", "TAHOE") options.set_capability("browserVersion", "31") options.set_capability("tb:binary_location", "testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app") options.set_capability("tb:app", "https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-darwin-arm64-1.0.0.zip") # Initialize the WebDriver connection to TestingBot driver = webdriver.Remote( command_executor="https://key:secret@hub.testingbot.com/wd/hub", options=options ) try: # Generate two random numbers between 0 and 9 num1 = random.randint(0, 9) num2 = random.randint(0, 9) # Find elements on the page first_number = driver.find_element(By.ID, f"btn-{num1}") second_number = driver.find_element(By.ID, f"btn-{num2}") plus = driver.find_element(By.ID, "btn-plus") equal = driver.find_element(By.ID, "btn-equal") # Perform operations first_number.click() plus.click() second_number.click() equal.click() # Get the result result_element = driver.find_element(By.ID, "calc-display") result_text = result_element.text # Verify the result assert (num1 + num2) == int(result_text), "Test failed" finally: driver.quit() ``` ```javascript const { Builder, By } = require('selenium-webdriver'); const { Options } = require('selenium-webdriver/chrome'); (async function example() { let driver; try { // Set capabilities let caps = { browserName: 'electron', platformName: 'TAHOE', browserVersion: '31', 'tb:binary_location': 'testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app', 'tb:app': 'https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-darwin-arm64-1.0.0.zip' }; driver = await new Builder() .usingServer('https://key:secret@hub.testingbot.com/wd/hub') .withCapabilities(caps) .build(); // Generate two random numbers between 0 and 9 const num1 = Math.floor(Math.random() * 10); const num2 = Math.floor(Math.random() * 10); const firstNumber = await driver.findElement(By.id(`btn-${num1}`)); const secondNumber = await driver.findElement(By.id(`btn-${num2}`)); const plus = await driver.findElement(By.id('btn-plus')); const equal = await driver.findElement(By.id('btn-equal')); // Perform operations await firstNumber.click(); await plus.click(); await secondNumber.click(); await equal.click(); // Get the result const resultElement = await driver.findElement(By.id('calc-display')); const resultText = await resultElement.getText(); // Verify the result if ((num1 + num2) !== parseInt(resultText)) { throw new Error('Test failed'); } } catch (error) { console.error('Error during test execution:', error); } finally { if (driver) { await driver.quit(); } } })(); ``` ```csharp using OpenQA.Selenium; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Remote; using System; class SeleniumTest { static void Main(string[] args) { try { // Set capabilities var options = new ChromeOptions(); options.BrowserName = "electron"; options.PlatformName = "TAHOE"; options.BrowserVersion = "31"; // Use AddAdditionalOption (Selenium 4) so these vendor caps land at the top level, // not nested under goog:chromeOptions like AddAdditionalChromeOption would. options.AddAdditionalOption("tb:binary_location", "testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app"); options.AddAdditionalOption("tb:app", "https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-darwin-arm64-1.0.0.zip"); IWebDriver driver = new RemoteWebDriver( new Uri("https://key:secret@hub.testingbot.com/wd/hub"), options ); // Generate two random numbers between 0 and 9 Random random = new Random(); int num1 = random.Next(0, 10); int num2 = random.Next(0, 10); IWebElement firstNumber = driver.FindElement(By.Id($"btn-{num1}")); IWebElement secondNumber = driver.FindElement(By.Id($"btn-{num2}")); IWebElement plus = driver.FindElement(By.Id("btn-plus")); IWebElement equal = driver.FindElement(By.Id("btn-equal")); // Perform operations firstNumber.Click(); plus.Click(); secondNumber.Click(); equal.Click(); // Get the result IWebElement resultElement = driver.FindElement(By.Id("calc-display")); string resultText = resultElement.Text; // Verify the result if ((num1 + num2) != int.Parse(resultText)) { throw new Exception("Test failed"); } // Quit the driver driver.Quit(); } catch (Exception ex) { Console.WriteLine($"Error during test execution: {ex.Message}"); } } } ``` The example above will run the TestingBot Demo Electron app on a remote macOS machine. You can also choose to run the demo app on a remote Windows machine inside TestingBot's grid, please see the capabilities below. ```ruby options = Selenium::WebDriver::Options.chrome options.browser_name = "electron" options.platform_name = "WIN10" options.browser_version = "31" options.add_option("tb:binary_location", "testingbot-electron-demo-app.exe") options.add_option("tb:app", "https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-win32-x64-1.0.0.zip") ``` ## Configuring Electron tests To configure your Electron test with TestingBot, you need to supply 3 capabilities: - [app capability](https://testingbot.com#capability-app) - [binary\_location capability](https://testingbot.com#capability-binary) - Electron version capability Please use the `version` or `browserVersion` capability to indicate to TestingBot which version of Electron was used to create the final app that you are testing. TestingBot needs to know which version was used, so that we can map it to the Chromium version that is being used for that particular Electron version. For example, if you created your Electron app with Electron 42, you would need to provide `browserVersion: 42` as a capability. ## Electron Test Results Test results for each Electron test will be available in the TestingBot member dashboard. For each Electron test, you will get access to a video recording, generated log files and additional meta-data. These results are also available through the TestingBot [REST-API](https://testingbot.com/support/api) or by using one of the [TestingBot CI/CD plugins](https://testingbot.com/support/integrations/ci-cd). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/electron/playwright # Electron App Testing with Playwright [Electron](https://electronjs.org/) apps embed a Chromium engine, which means [Playwright](https://playwright.dev/) can automate them through the Chrome DevTools Protocol (CDP). TestingBot starts your Electron app on a remote Windows or macOS machine and gives you a WebSocket endpoint that Playwright connects to with `chromium.connectOverCDP()`. Your Playwright test code runs on your own machine or CI server, while the Electron app itself runs in the TestingBot cloud, on these platforms: - Windows 11 - Windows 10 - macOS Tahoe - macOS Sequoia - macOS Sonoma - macOS Ventura - macOS Monterey All Electron versions from version 16 to the most recent version are supported. If you prefer WebDriver (Selenium) over Playwright, please see [Electron testing with WebDriver](https://testingbot.com/support/app-automate/electron). TestingBot has created a [demo Electron app](https://github.com/testingbot/testingbot-electron-demo-app), which acts as a simple calculator, to showcase automated Electron app testing. The examples below run against this demo app. ## Prepare your Electron app TestingBot needs to be able to download your Electron app as a `zip` file. You can host this zip file yourself, for example on AWS S3 or GitHub Releases, or upload it to [TestingBot Storage](https://testingbot.com/support/api/storage#upload), which returns a `tb://` URL you can use instead. Next to the URL of the zip file, you also need to know the path of the actual Electron binary inside the zip file. This is the `binary_location` parameter, so TestingBot knows which file to launch. For the demo app on macOS this is: ```bash testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app ``` On Windows, it is simply the executable in the zip file: ```bash testingbot-electron-demo-app.exe ``` See [App management](https://testingbot.com/support/app-automate/electron#capability-app) on the WebDriver Electron page for more details about uploading your app. ## Run your first test Install `playwright-core`, connect with `chromium.connectOverCDP()` and pass the Electron parameters in the WebSocket URL. Once connected, your Electron app's window is available as the first page of the first browser context. The `browserVersion` parameter must be the major Electron version your app was built with. The demo app was built with Electron 31, so the examples below use `browserVersion=31`. ```bash npm i playwright-core ``` [macOS](https://testingbot.com#) [Windows](https://testingbot.com#) ```javascript const { chromium } = require('playwright-core'); const app = 'https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-darwin-arm64-1.0.0.zip'; const binaryLocation = 'testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app'; const wsEndpoint = 'wss://cloud.testingbot.com' + '?key=api_key&secret=api_secret' + '&browserName=electron' + '&browserVersion=31' + '&platform=TAHOE' + `&app=${encodeURIComponent(app)}` + `&binary_location=${encodeURIComponent(binaryLocation)}` + `&name=${encodeURIComponent('My First Electron Test')}`; (async () => { // Starting the remote machine and downloading your app can take a while const browser = await chromium.connectOverCDP(wsEndpoint, { timeout: 300000 }); const context = browser.contexts()[0]; const page = context.pages()[0]; // the Electron app window // Calculate 3 + 4 in the demo calculator app await page.click('#btn-3'); await page.click('#btn-plus'); await page.click('#btn-4'); await page.click('#btn-equal'); const result = await page.textContent('#calc-display'); console.log(`3 + 4 = ${result}`); await browser.close(); })(); ``` ```javascript const { chromium } = require('playwright-core'); const app = 'https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-win32-x64-1.0.0.zip'; const binaryLocation = 'testingbot-electron-demo-app.exe'; const wsEndpoint = 'wss://cloud.testingbot.com' + '?key=api_key&secret=api_secret' + '&browserName=electron' + '&browserVersion=31' + '&platform=WIN10' + `&app=${encodeURIComponent(app)}` + `&binary_location=${encodeURIComponent(binaryLocation)}` + `&name=${encodeURIComponent('My First Electron Test')}`; (async () => { // Starting the remote machine and downloading your app can take a while const browser = await chromium.connectOverCDP(wsEndpoint, { timeout: 300000 }); const context = browser.contexts()[0]; const page = context.pages()[0]; // the Electron app window // Calculate 3 + 4 in the demo calculator app await page.click('#btn-3'); await page.click('#btn-plus'); await page.click('#btn-4'); await page.click('#btn-equal'); const result = await page.textContent('#calc-display'); console.log(`3 + 4 = ${result}`); await browser.close(); })(); ``` Make sure to use `chromium.connectOverCDP()` and the `wss://cloud.testingbot.com` endpoint for Electron tests. The `wss://cloud.testingbot.com/playwright` endpoint used for [Playwright browser testing](https://testingbot.com/support/web-automate/playwright) does not support Electron apps. ## Playwright Test You can use the same connection with [Playwright Test](https://testingbot.com/support/web-automate/playwright/playwright-test), Playwright's own test runner. Connect in a `beforeAll` hook or fixture and run your suite with `npx playwright test`. ```javascript const { test, expect, chromium } = require('@playwright/test'); const app = 'https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-darwin-arm64-1.0.0.zip'; const binaryLocation = 'testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app'; const wsEndpoint = 'wss://cloud.testingbot.com' + '?key=api_key&secret=api_secret' + '&browserName=electron' + '&browserVersion=31' + '&platform=TAHOE' + `&app=${encodeURIComponent(app)}` + `&binary_location=${encodeURIComponent(binaryLocation)}` + `&name=${encodeURIComponent('Electron Playwright Test')}`; test('calculator adds two numbers', async () => { test.setTimeout(600000); // remote machine startup can take a few minutes const browser = await chromium.connectOverCDP(wsEndpoint, { timeout: 300000 }); const page = browser.contexts()[0].pages()[0]; await page.click('#btn-3'); await page.click('#btn-plus'); await page.click('#btn-4'); await page.click('#btn-equal'); await expect(page.locator('#calc-display')).toHaveText('7'); await browser.close(); }); ``` ## Python Playwright for Python offers the same `connect_over_cdp()` API, so you can test your Electron app from Python as well. ```bash pip install playwright ``` ```python from urllib.parse import quote from playwright.sync_api import sync_playwright app = "https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-darwin-arm64-1.0.0.zip" binary_location = "testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app" ws_endpoint = ( "wss://cloud.testingbot.com" "?key=api_key&secret=api_secret" "&browserName=electron" "&browserVersion=31" "&platform=TAHOE" f"&app={quote(app, safe='')}" f"&binary_location={quote(binary_location, safe='')}" "&name=Electron%20Playwright%20Python" ) with sync_playwright() as p: # Starting the remote machine and downloading your app can take a while browser = p.chromium.connect_over_cdp(ws_endpoint, timeout=300000) page = browser.contexts[0].pages[0] # the Electron app window page.click("#btn-3") page.click("#btn-plus") page.click("#btn-4") page.click("#btn-equal") result = page.text_content("#calc-display") print(f"3 + 4 = {result}") browser.close() ``` ## Connection parameters All parameters are passed as query parameters in the WebSocket URL. Make sure to URL-encode the values, especially the `app` URL and `binary_location` path. Parameter | Description || `key` and `secret` | Your TestingBot [API credentials](https://testingbot.com/members/user/edit). | | `browserName` | Must be `electron`. | | `browserVersion` | The major Electron version your app was built with, for example `42`. TestingBot uses this to match the correct Chromium version. | | `platform` | The operating system to run your app on, for example `TAHOE`, `SEQUOIA`, `WIN10` or `WIN11`. | | `app` | URL of the zip file containing your Electron app, or a `tb://` URL from [TestingBot Storage](https://testingbot.com/support/api/storage#upload). | | `binary_location` | Path of the Electron binary inside the zip file. | | `name` | Optional name for the test, shown in the [TestingBot dashboard](https://testingbot.com/members). | | `build` | Optional build identifier to group multiple tests together. | ## Mark tests as passed or failed Playwright does not report test success or failure to TestingBot by itself. You can use the `testingbot_executor` custom command from within your test to mark the session as passed or failed in the TestingBot dashboard. ```javascript try { await expect(page.locator('#calc-display')).toHaveText('7'); // Mark the test as passed on TestingBot await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({ action: 'setSessionStatus', arguments: { passed: true, reason: 'Calculator sum matched' } })}`); } catch (err) { // Mark the test as failed on TestingBot await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({ action: 'setSessionStatus', arguments: { passed: false, reason: 'Calculator sum did not match' } })}`); throw err; } ``` ## Supported Playwright features Playwright talks to your Electron app through the Chrome DevTools Protocol, which supports everything that operates on browser contexts and pages: - Locators, clicking, typing and all other page interactions - Assertions with `expect()` - Screenshots - Evaluating JavaScript in the renderer process with `page.evaluate()` - Network inspection of requests made by your app's windows Because your app runs on a remote machine, the Electron-specific APIs from Playwright's local `_electron.launch()` mode are not available. This means no `electronApp.evaluate()` access to the Electron main process, such as `app` or `BrowserWindow` modules. If you need to assert on main process behavior, expose that state to the renderer process, for example via IPC, and read it with `page.evaluate()`. ## Test Results Every Electron Playwright test appears in the TestingBot member dashboard. For each test you get access to a video recording, log files and additional meta-data. These results are also available through the TestingBot [REST-API](https://testingbot.com/support/api) or by using one of the [TestingBot CI/CD plugins](https://testingbot.com/support/integrations/ci-cd). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/earlgrey # EarlGrey Testing TestingBot will release testing with EarlGrey in Q2 of 2026. Run your EarlGrey tests on physical Android devices in the cloud, on Samsung Galaxy devices, Pixels, Xiaomi and other Android devices. If you are interested in participating in our beta program, please contact us. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/maestro # Maestro Cloud Testing TestingBot provides a grid of real iOS and Android devices, plus iOS simulators and Android emulators, ready to run your Maestro flows. Connect your existing Maestro YAML flows to the TestingBot cloud and run them in parallel from your machine or your CI pipeline. Maestro is a mobile UI testing framework, which supports testing on both iOS and Android. You write flows, which are story-like tests, where you describe a specific flow in your mobile application. For example, you could have a login flow, or a checkout flow. Maestro is a relatively new test framework and offers several nice features, such as: - Built-in tolerance to flakiness: various mechanisms are added to try and prevent flaky tests, for example locators that have changed or moved positions. - Automatic waits: Maestro will automatically wait for certain conditions, which means you do not have to worry about delays, or adding sleep commands to your tests. - Declarative syntax, which is defined in Flow (`yaml`) files. ## Why TestingBot Cloud? TestingBot has been offering cloud-based testing since 2012 and has built a robust solution for Maestro cloud testing. Compared to other vendors and Maestro cloud, we offer the following improved features: - Run on any iOS version, both physical iOS devices and iOS simulators. - Test on **physical Android devices** , not just emulators (though that's also certainly possible) - No execution limit: your tests can run as long as they need - Apply various [options](https://testingbot.com/support/app-automate/maestro/options) such as setting orientation, geolocation options and more **New to TestingBot?** Maestro users get a 30-day free trial. [Start your trial](https://testingbot.com/users/sign_up) or see the [Maestro feature overview](https://testingbot.com/features/automation/maestro). ## TestingBot CLI The easiest way to run Maestro tests on TestingBot is using our CLI tool. It handles uploading your app, flows and running tests in a single command. https://www.youtube.com/embed/MMWCkBgXFOU ### Installation ```bash npm install -g @testingbot/cli ``` ### Authentication Authenticate using this command: ```bash testingbot login ``` This opens your browser for authentication. After logging in, your credentials are saved to `~/.testingbot` and you can start using the CLI. #### Other Authentication Methods - **Command-line options** : `--api-key` and `--api-secret` - **Environment variables** : `TB_KEY` and `TB_SECRET` - **Config file** : Create `~/.testingbot` with content `key:secret` ### Quick Start Run Maestro tests with a single command: ```bash testingbot maestro app.apk flow.yml --device "Pixel 8" --deviceVersion "14" ``` ```bash testingbot maestro app.ipa flow.yml --device "iPhone 16" --real-device ``` The CLI will: 1. Upload your app to TestingBot, from a local file or a download URL such as an EAS Build link (`--app-url`), or reuse an earlier upload with `--app-binary-id` 2. Upload and zip your flow files 3. Start the test run, on one device or a whole [device matrix](https://testingbot.com/support/app-automate/maestro/options#device-matrix) 4. Show real-time progress and results, and exit `0`, `2` (flows failed) or `1` (error) For pipelines there is `--json` output, `--async` with `testingbot status --id --wait` and `testingbot artifacts` to come back for results, a [GitHub Action](https://testingbot.com/support/app-automate/maestro/ci-cd/github-actions), [GitHub PR status checks](https://testingbot.com/support/app-automate/maestro/ci-cd/github-pr-checks) and [Expo / EAS](https://testingbot.com/support/app-automate/maestro/ci-cd/expo-eas) support. See [CI/CD integration](https://testingbot.com/support/app-automate/maestro/ci-cd). Coming from Maestro Cloud? The [migration guide](https://testingbot.com/support/app-automate/maestro/migrate-from-maestro-cloud) maps every `maestro cloud` flag to its TestingBot equivalent. ## Upload Mobile App You will need to upload a mobile app to run tests with Maestro. The format of this file depends on which device type you'd like to test: - **iOS physical devices/simulators** : You can upload your `.ipa` or `.zip` file (for simulators, see [here how to prepare this zip file](https://testingbot.com/support/app-automate/help/prepare#simulator)). - **Android physical devices/emulators** : Upload the `.apk` file, which TestingBot will automatically install on the Android device. [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) When using the CLI, the app upload is handled automatically as part of the `testingbot maestro` command. Simply specify your app file as the first argument: ```bash testingbot maestro /path/to/app/file/sample-debug.apk ./flows ``` Supported app formats: - **Android** : `.apk` - **iOS Physical Device** : `.ipa` - **iOS Simulator** : `.app`, `.zip` (zipped .app bundle) ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/maestro/app" \ -F "file=@/path/to/app/file/sample-debug.zip" ``` The API response will include an `id` response, which you can use in the other API calls. ```json { "id": 4012 } ``` ## Upload Maestro Flows To upload the Flows zip file, please follow the example below. [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) When using the CLI, flow files are automatically zipped and uploaded. You can specify: - A directory containing flow files - Individual `.yaml` or `.yml` files - A pre-zipped `.zip` file - Glob patterns (e.g., `./flows/**/*.yaml`) ```bash # Single directory testingbot maestro app.apk ./flows # Multiple directories testingbot maestro app.apk ./flows/smoke ./flows/regression # Individual flow files testingbot maestro app.apk login.yaml checkout.yaml # Pre-zipped flows testingbot maestro app.apk flows.zip ``` Maestro Flow files are `.yaml` files which contain the actions that need to be performed. You can zip these together and upload to TestingBot by using the following command: ```bash zip -r flows.zip *.yaml ``` Make sure all the necessary flows and sub flows are included in the zip file. ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/tests" \ -F "file=@/path/to/app/file/flows.zip" ``` Replace the `:id` with the identifier you received during the app upload call. (In this example 4012) Calling this multiple times with the same `:id` will overwrite the previously uploaded Flows zip file. ### Zip structure Maestro needs to know which flows it should run. If there are flow files (`.yaml` files) in the top-level directory of the zip file, it will run these flow files. You can also include a [config.yaml](https://testingbot.com/support/app-automate/maestro/options#config) file in the top-level directory of the zip file, to indicate which flows should be tested. For example, you can use the `**` annotation to indicate you want to run all flows, even if they are in separate sub-directories. ```yaml flows: - "**" ``` More information is available in the [Maestro Docs](https://docs.maestro.dev/cli/test-suites-and-reports#controlling-what-tests-to-include). ## Run Maestro tests Once you've uploaded both your app and Flows zip file, you can run the Maestro tests on the TestingBot cloud. Specify one or more capabilities, to indicate on which iOS simulators or Android devices you want to run your tests on. [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) Use the CLI to run tests with device selection options: ```bash # Basic usage with device selection testingbot maestro app.apk flow.yml --device "Pixel 8" --deviceVersion "14" # iOS simulator testingbot maestro app.zip ./flows --platform iOS --device "iPhone 16" --deviceVersion "26.0" # iOS physical device testingbot maestro app.ipa flow.yml --device "iPhone 16" --real-device # Android real device testingbot maestro app.apk flow.yml --device "Galaxy S24" --real-device # With additional options testingbot maestro app.apk ./flows \ --device "Pixel 8" \ --deviceVersion "14" \ --name "Smoke Tests" \ --build "v2.1.0" # With CI/CD integration (Git metadata) testingbot maestro app.apk ./flows \ --device "Pixel 8" \ --commit-sha "$COMMIT_SHA" \ --pull-request-id "$PR_NUMBER" \ --repo-owner "myorg" \ --repo-name "myapp" ``` ### Device Options | Option | Description | | --- | --- | | `--device ` | Device name (e.g., "Pixel 9", "iPhone 16") | | `--platform ` | Platform: Android or iOS | | `--deviceVersion ` | OS version (e.g., "14", "18.0") | | `--real-device` | Use a real device instead of emulator/simulator | See the [Options page](https://testingbot.com/support/app-automate/maestro/options) for more configuration options. Select a device configuration below, then use the generated cURL command: ![OS Selected](https://testingbot.com/assets/environments/svg/ios-383468addf160fa18d0e431f529420739d7f7d1206175f682fead627d2e99a52.svg) iOS 18.6 › iPhone 16 Loading environments... Please wait while we load the available browsers and platforms. Replace the `:id` with the identifier you received during the app upload call. (In this example 4012) When running Maestro tests on physical devices, a device could be occupied at times. You might want to use regex/wildcard patterns to target a broad range of devices. We offer special parameters which you can use to allocate a device: Regex Input | Result || `".*Galaxy.*"` | This will allocate any of the available Galaxy devices (phone or tablet) | | `"Pixel.*"` | This will allocate any of the available Pixel devices. | | `".*Galaxy.*\|.*Pixel.*"` | This will allocate any available Galaxy or Pixel device. Use `\|` to combine patterns; a comma is matched literally and will find no device. | | `"*"` | This will allocate a random available device, you can use this with the `deviceName` or `version`. | The `version` capability accepts the same patterns, and additionally a comparison against an OS version. This is useful when you do not want to pin a run to a version that may be retired. Version Input | Result || `"14"` | Any 14.x version. | | `"18.6"` | Exactly this version. | | `">=14"` | The newest available version of 14 or higher. `>`, `<`, `<=` and `=` work too. | | `"1[34].*"` | Any 13.x or 14.x version. | | `"*"` | Any available version. | ## View Maestro Results The results of your Maestro tests will be available in the [TestingBot dashboard](https://testingbot.com/members). Each test result contains a video, test logs and other meta data generated during the test run. Maestro flows will be displayed, together with the steps that were taken during each flow. ![Maestro results](https://testingbot.com/assets/support/maestro/results-78a1591a85866ec01fa9a661652d9fd4cf0b24989743d73756e8a991157dd7be.png) You can also use the CLI or an API call to get back the result from the run(s): [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) By default, the CLI waits for test completion and displays real-time progress: ```bash testingbot maestro app.apk ./flows --device "Pixel 8" ``` The CLI will show: - Upload progress for app and flows - Device allocation status - Live output from Maestro flows - Final pass/fail status ### Async Mode Use `--async` to start tests without waiting for results, then come back for them with `status` and `artifacts`: ```bash testingbot maestro app.apk ./flows --device "Pixel 8" --async # Tests started in async mode. Project ID: 1234 testingbot status --id 1234 --wait testingbot artifacts --id 1234 --report junit --report-output-dir ./reports ``` ### JSON Output Use `--json` to get every run and flow result as one JSON document on stdout, or `--json-file` to write it next to the normal output: ```bash testingbot maestro app.apk ./flows --device "Pixel 8" --json | jq .outcome ``` ### Download Artifacts Download test artifacts (logs, screenshots, video) after completion: ```bash testingbot maestro app.apk ./flows --device "Pixel 8" \ --download-artifacts \ --artifacts-output-dir ./artifacts ``` ```bash curl -u api_key:api_secret \ "https://api.testingbot.com/v1/app-automate/maestro/:project_id" ``` Replace the `:project_id` with the identifier you received during the app upload call. (In this example 4012) The response will contain a `success` boolean, indicating whether the test passed or failed. **Example response:** ```json {"runs":[{"id":14019,"status":"DONE","capabilities":{"version":"15","deviceName":"Pixel 9","platformName":"Android"},"success":0,"report":"...","created_at":"2025-08-26T10:30:42.000Z","test":{"sessionId":"0255832ee646e7620dc7d44917293ccf","environment":{"name":"android","os":"Pixel 9 - 15.0","version":"15.0"}}}],"success":false,"version":"1.0","completed":true} ``` You can poll this request to check if the tests have finished running. The `completed` boolean will be set to true when all tests have finished. ### JUnit Report All Maestro tests on TestingBot will by default run with a JUnit reporter. This report is accessible programmatically: [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) Download the JUnit report automatically after test completion: ```bash # Download JUnit XML report testingbot maestro app.apk ./flows --device "Pixel 8" \ --report junit \ --report-output-dir ./reports ``` The report will be saved to the specified directory with a filename based on the test run. `--report` also accepts `html`, `html-detailed` and `allure`. ```bash curl -u api_key:api_secret \ "https://api.testingbot.com/v1/app-automate/maestro/:project_id/:run_id/junit_report" ``` Replace the `:project_id` with the identifier you received during the app upload call. (In this example 4012) Replace the `:run_id` with the identifier for this run. The response will contain a `junit_report` string value that contains the XML formatted JUnit report. ### Webhooks Because the Maestro tests run asynchronously, you might want to get notified when a test run has finished. You can poll the [results API](https://testingbot.com#results) or use the [Webhook integration](https://testingbot.com/support/integrations/webhooks) or [Slack integration](https://testingbot.com/support/integrations/slack) to get notified when a test run has finished. ## Example Maestro Flow Below is a simple example Flow, where we input two values in the TestingBot example app: - [Android Sample .apk](https://testingbot.com/appium/sample.apk) - [iOS Simulator Sample .zip](https://testingbot.com/appium/sample.zip) - [iOS Physical Sample .ipa](https://testingbot.com/appium/sample.ipa) ```yaml appId: org.reactjs.native.example.SampleApp --- - launchApp - tapOn: id: inputA - longPressOn: id: inputA - tapOn: text: "Select All" - eraseText - inputText: "10" - tapOn: id: inputB - longPressOn: id: inputB - tapOn: text: "Select All" - eraseText - inputText: "5" - tapOn: id: sum - assertVisible: text: "15" ``` ## Maestro Parallel Testing The TestingBot physical and virtual device grid is optimised to run many tests concurrently. You can run multiple Maestro flows simultaneously (in parallel), which will drastically shorten your total test duration. Depending on your subscription plan, you will be able to run 1 or more Maestro flows at the same time. If you exceed the threshold, additional Maestro flows will be queued until a slot becomes available. By default, each flow runs in a separate session. Make sure each flow can run isolated. If you don't want this, you can use the [--shard-split 1](https://testingbot.com#shard-split) option to run all flows in the same session. For example, let's assume you have an Automation Pro plan with 2 concurrent tests. And you want to run 10 Maestro tests, or run the same Maestro test on 10 different device configurations: - 2 Maestro tests will run simultaneously - The other 8 are waiting (queued) until one of the first two finishes - When a slot frees up, the next Maestro test in the queue will start to run - Finally, when all tests have run, you will see the results in the TestingBot dashboard Currently there is no limit to the amount of Maestro tests you can add to the queue. ## Grouping Flows When you have a large number of flows, you might want to split these flows across multiple test runs. This is also known as sharding. By default, each test runs separately in parallel on TestingBot. You can group flows together that should run in the same session by using the `--shard-split` option. [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash # Shard flows across 3 parallel sessions testingbot maestro app.apk ./flows \ --device "Pixel 8" \ --deviceVersion "14" \ --shard-split 3 ``` You can also decide to run all flows in the same session by using the `--shard-split 1` option. ```bash # Run all flows in a single session testingbot maestro app.apk ./flows \ --device "Pixel 8" \ --deviceVersion "14" \ --shard-split 1 ``` Use the `shardSplit` parameter in your API request to control how flows are grouped. ```bash # Shard flows across 3 parallel sessions curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \ -d '{"capabilities":[{"version":"14","deviceName":"Pixel 8","platformName":"Android"}], "shardSplit": 3}' \ -H "Content-Type: application/json" ``` To run all flows in a single session, set `shardSplit` to `1`: ```bash # Run all flows in a single session curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \ -d '{"capabilities":[{"version":"14","deviceName":"Pixel 8","platformName":"Android"}], "shardSplit": 1}' \ -H "Content-Type: application/json" ``` Replace the `:id` with the identifier you received during the app upload call. ## Integrations TestingBot offers various [integrations](https://testingbot.com/support/integrations) which allow for connecting your Maestro tests with external services. For example; you can receive automated Slack messages each time a Maestro test runs or fails, or you can integrate Maestro testing with any CI/CD system. - [App Center](https://testingbot.com/support/integrations/appcenter) - [Bitrise](https://testingbot.com/support/integrations/ci-cd/bitrise) - [GitHub Actions (Maestro action)](https://testingbot.com/support/app-automate/maestro/ci-cd/github-actions) - [GitHub PR Checks](https://testingbot.com/support/app-automate/maestro/ci-cd/github-pr-checks) - [Expo / EAS Workflows](https://testingbot.com/support/app-automate/maestro/ci-cd/expo-eas) - [Jenkins CI](https://testingbot.com/support/integrations/ci-cd/jenkins) - [Jira](https://testingbot.com/support/integrations/jira) - [Slack](https://testingbot.com/support/integrations/slack) - [TeamCity CI](https://testingbot.com/support/integrations/ci-cd/teamcity) - [Webhooks](https://testingbot.com/support/integrations/webhooks) ## API TestingBot provides a [REST-API](https://testingbot.com/support/app-automate/maestro/api) to retrieve information about your Maestro projects and runs. ## Frequently asked questions What is Maestro and how does it differ from Appium? Maestro is a declarative mobile UI test framework built by mobile.dev. You describe user flows in YAML files (login, checkout, etc.) and Maestro runs them on iOS and Android. Compared to Appium, Maestro has automatic waits, built-in flakiness tolerance and a much simpler syntax, but it does not implement the W3C WebDriver protocol so Appium tooling does not apply. How do I run Maestro tests on TestingBot? Install the [TestingBot CLI](https://testingbot.com#cli), upload your app and Maestro flow ZIP to [TestingBot Storage](https://testingbot.com/support/api/storage#upload), then run the flows with a single CLI command or via the [REST API](https://testingbot.com/support/app-automate/maestro/api). Flows run in parallel on real iPhones, iPads, Android phones and tablets, plus iOS simulators and Android emulators. Can I run Maestro on real iOS and Android devices? Yes. TestingBot supports Maestro on real physical iPhones, iPads, Android phones and tablets in our EU datacenter, in addition to iOS simulators and Android emulators. Target a specific device name or use wildcards like `".*Galaxy.*"` to allocate any available matching device. Why are my Maestro tests slow on real devices? By default Maestro flows run in parallel on TestingBot. When you pin a specific device, each session waits for that device to free up and complete its post-test clean cycle. Two fixes: (1) target a broader device pattern with wildcards or regex; (2) run all flows in one session with `--shard-split 1` (CLI) or `shardSplit: 1` (API). See the [Grouping Flows section](https://testingbot.com#shard-split) for details. Can I run Maestro tests from CI/CD? Yes. TestingBot has dedicated [CI/CD integration docs](https://testingbot.com/support/app-automate/maestro/ci-cd) with examples for GitHub Actions, GitLab CI, Jenkins and CircleCI, plus exit codes, environment variables and build metadata. Can I pin a specific Maestro version? Yes. Pass the `maestroVersion` option to pin to an exact Maestro release. See the [Maestro version option](https://testingbot.com/support/app-automate/maestro/options#version) for current supported versions. How do I fetch Maestro results, video and logs programmatically? Use the [Maestro REST API](https://testingbot.com/support/app-automate/maestro/api) to list runs, fetch per-flow results, download video and screenshot artifacts, and stream Maestro step logs. The dashboard exposes the same data interactively. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/maestro/api # Maestro API Overview TestingBot provides a REST API to upload apps, run Maestro flows, and retrieve information about your [Maestro projects and runs](https://testingbot.com/support/app-automate/maestro). In the documentation below, a Maestro project using a `:project_id` consists of a mobile application that was uploaded together with a zipped set of Maestro flow files. A Maestro project can have multiple test runs, depending on how many device configurations (capabilities) you specified when you [started the tests](https://testingbot.com/support/app-automate/maestro/api#run). All API endpoints require authentication using your TestingBot API key and secret, which you can obtain from the Member area. | Base URL | Authentication | | --- | --- | | `https://api.testingbot.com/v1/app-automate/maestro` | HTTP Basic Auth (API key:secret) | ## CLI Alternative For most use cases, we recommend using the [TestingBot CLI](https://testingbot.com/support/app-automate/maestro#cli) instead of the REST API directly. The CLI handles uploading your app, flows and running tests in a single command: ```bash # Install the CLI npm install -g @testingbot/cli # Authenticate testingbot login # Run Maestro tests (handles upload + run automatically) testingbot maestro app.apk ./flows --device "Pixel 8" --deviceVersion "14" ``` The CLI provides: - Automatic app and flow uploads - Real-time progress and streaming output - Automatic report downloads - Simpler authentication Use the REST API directly when you need programmatic access or custom integration workflows. ## Upload App Upload your Android (.apk/.aab) or iOS (.ipa/.app) application to TestingBot. This creates a new Maestro project and returns a project ID. | Method | Endpoint | | --- | --- | | `POST` | `/app-automate/maestro/app` | ### Parameters | Name | Type | Required | Description | | --- | --- | --- | --- | | `file` | file | Yes | The app file to upload (.apk, .aab, .ipa, .app or .zip) | [cURL](https://testingbot.com#) ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/maestro/app" \ -F "file=@/path/to/your/app.apk" ``` ### Response ```json { "id": 17876, "app": { "app_url": "https://...", "icon_url": "https://...", "app_version": "1.0.0", "bundle_id": "com.example.app" } } ``` Save the returned `id` - you'll need it for uploading flows and running tests. ## Reuse App Create a new Maestro project that shares the app binary of a project uploaded earlier, so you can run new flows against the same build without uploading it again. The source project is left untouched. This is what the CLI's `--app-binary-id` flag calls. | Method | Endpoint | | --- | --- | | `POST` | `/app-automate/maestro/app/:id/reuse` | ### Parameters | Name | Type | Required | Description | | --- | --- | --- | --- | | `id` | integer | Yes | The project ID whose app should be reused | [cURL](https://testingbot.com#) ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/maestro/app/17876/reuse" ``` ### Response ```json { "id": 17901, "source_id": 17876, "platform": "Android", "app": { "app_url": "https://...", "icon_url": "https://...", "app_version": "1.0.0", "bundle_id": "com.example.app" } } ``` ### Error Responses | Status Code | Description | | --- | --- | | `404` | No project with this ID on your account | | `410` | The app of this project is no longer in storage. Upload the app again. | Use the returned `id` as the project for [Upload Maestro Flows](https://testingbot.com#uploadFlows) and [Run Maestro Tests](https://testingbot.com#run). `platform` is derived from the stored app so you know which device capabilities apply. ## Upload Maestro Flows Upload your Maestro flow files (as a .zip archive) to an existing project. This replaces any previously uploaded flows. | Method | Endpoint | | --- | --- | | `POST` | `/app-automate/maestro/:id/tests` | ### Parameters | Name | Type | Required | Description | | --- | --- | --- | --- | | `id` | integer | Yes | The project ID returned from the app upload | | `file` | file | Yes | A .zip file containing your Maestro flow YAML files | [cURL](https://testingbot.com#) ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/tests" \ -F "file=@/path/to/flows.zip" ``` ### Response ```json { "id": 17876, "flows": [ { "id": 123, "name": "login_flow" } ] } ``` ## Run Maestro Tests Start running your Maestro flows on one or more devices. You can specify multiple device configurations to run tests in parallel; every capability becomes its own run (this is how the CLI's `--device-matrix` works). All capabilities are validated first: if any one is not a valid device and OS combination, the request is rejected as a whole and no run is created. | Method | Endpoint | | --- | --- | | `POST` | `/app-automate/maestro/:id/run` | ### Parameters | Name | Type | Required | Description | | --- | --- | --- | --- | | `id` | integer | Yes | The project ID | | `capabilities` | array | Yes | Array of device configurations to run tests on | | `maestroOptions` | object | No | Additional Maestro options (version, env, flows, tags). See [Test Options](https://testingbot.com/support/app-automate/maestro/options). | | `shardSplit` | integer | No | Group flows into N chunks, each chunk running on a separate device. When omitted, each flow runs in its own session on the same device. | | `metadata` | object | No | CI/CD metadata stored with the run (`commitSha`, `pullRequestId`, `pullRequestUrl`, `repoOwner`, `repoName`, `branch`, plus a free-form `custom` object). Displayed in the TestingBot dashboard. When `repoOwner`, `repoName` and a full 40-character `commitSha` are present and the [TestingBot GitHub App](https://testingbot.com/support/integrations/ci-cd/github-pr-checks#maestro) is installed for that repository, a status check is posted on the commit and the response carries a `ci_check` object. | [cURL](https://testingbot.com#) ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \ -d '{"capabilities":[{"platformName":"Android","version":"14","deviceName":"Pixel 8"}]}' \ -H "Content-Type: application/json" ``` ### Response ```json { "success": true, "id": 17876, "runs": [ { "id": 18809, "capabilities": { "platformName": "Android", "version": "14", "deviceName": "Pixel 8" }, "flows": [ { "id": 123, "name": "login_flow", "report": "...", "requested_at": "2025-11-28T09:15:00.000Z", "completed_at": "2025-11-28T09:16:00.000Z", "status": "DONE", "test_case_id": 456 } ] } ] } ``` When a GitHub check was registered for the run, the response also includes: ```json "ci_check": { "provider": "github", "repo": "acme/app", "commit_sha": "<40-hex sha>" } ``` ### Error Responses | Status Code | Description | | --- | --- | | `200` with `success: false` | One or more capabilities are not a valid device and OS combination. `errors` lists them and `runs` is empty; nothing was started. | | `402` | Insufficient credits to run this test. Please upgrade your account to continue running Maestro tests. | ## List Maestro Projects Retrieve a paginated list of all your Maestro projects. | Method | Endpoint | | --- | --- | | `GET` | `/app-automate/maestro` | ### Query Parameters | Name | Type | Default | Description | | --- | --- | --- | --- | | `offset` | integer | 0 | Number of records to paginate | | `count` | integer | 10 | Number of records to return | [cURL](https://testingbot.com#) ```bash curl -u api_key:api_secret \ "https://api.testingbot.com/v1/app-automate/maestro" ``` ### Response ```json { "data": [ { "id": 17876, "name": "Maestro Project 17876", "created_at": "2025-11-28T08:12:31.000Z", "updated_at": "2025-11-28T08:12:31.000Z", "completed": false, "app": { "app_url": "https://...", "icon_url": "https://...", "app_version": "2.7.50420", "bundle_id": "org.wikipedia" }, "flows_url": "https://...", "flows": [ { "id": 123, "name": "login_flow", } ], "runs": [18809, 18854, 18863, 18866] } ], "meta": { "offset": 0, "count": 10, "total": 25 } } ``` ## List Maestro Runs Retrieve a paginated list of all your Maestro runs across every project, newest first. This is a lightweight endpoint: it returns run metadata only and does not inflate the (potentially large) flow reports, so it is well suited to dashboards and CI lookups. For a single run's full results and reports, use [Get Run Info](https://testingbot.com#runInfo). | Method | Endpoint | | --- | --- | | `GET` | `/app-automate/maestro/runs` | ### Query Parameters | Name | Type | Default | Description | | --- | --- | --- | --- | | `offset` | integer | 0 | Number of records to paginate | | `count` | integer | 10 | Number of records to return | [cURL](https://testingbot.com#) ```bash curl -u api_key:api_secret \ "https://api.testingbot.com/v1/app-automate/maestro/runs?count=10&offset=0" ``` ### Response ```json { "data": [ { "id": 18809, "project_id": 17876, "name": "maestro-android-PR12730-26648200708", "status": "DONE", "success": 1, "capabilities": { "platformName": "Android", "version": "14", "deviceName": "Pixel 8" }, "created_at": "2025-11-28T09:15:00.000Z", "completed_at": "2025-11-28T09:16:00.000Z" } ], "meta": { "offset": 0, "count": 10, "total": 25 } } ``` The `status` and `success` fields reflect the run's last-known state. A run is finished once `status` is `DONE` or `FAILED`. ## Find Run by Build Name Look up runs by their build name (the `name` capability you set when [starting the tests](https://testingbot.com#run)). This lets you resolve a build name straight to its `project_id` and `run_id` in a single request, instead of listing projects and scanning each one. A single build can fan out to multiple runs (for example one run per device configuration), so this endpoint always returns an array of every matching run. | Method | Endpoint | | --- | --- | | `GET` | `/app-automate/maestro/runs/:build_name` | ### Parameters | Name | Type | Required | Description | | --- | --- | --- | --- | | `build_name` | string | Yes | The build name to look up, matched against the run's `name`. Remember to URL-encode names that contain spaces or special characters. | [cURL](https://testingbot.com#) ```bash curl -u api_key:api_secret \ "https://api.testingbot.com/v1/app-automate/maestro/runs/maestro-android-PR12730-26648200708" ``` ### Response ```json { "data": [ { "id": 18809, "project_id": 17876, "name": "maestro-android-PR12730-26648200708", "status": "DONE", "success": 1, "capabilities": { "platformName": "Android", "version": "14", "deviceName": "Pixel 8" }, "created_at": "2025-11-28T09:15:00.000Z", "completed_at": "2025-11-28T09:16:00.000Z" } ] } ``` ### Error Responses | Status Code | Description | | --- | --- | | `404` | No run matches the supplied build name on your account. | Use the returned `project_id` and `id` (run ID) to fetch full results from [Get Run Info](https://testingbot.com#runInfo) or download the [JUnit Report](https://testingbot.com#report). ## Get Maestro Project Info Retrieve details about a specific Maestro project, including all its runs. | Method | Endpoint | | --- | --- | | `GET` | `/app-automate/maestro/:id` | [cURL](https://testingbot.com#) ```bash curl -u api_key:api_secret \ "https://api.testingbot.com/v1/app-automate/maestro/:id" ``` ### Response ```json { "runs": [ { "id": 18809, "status": "DONE", "capabilities": { "version": "16", "deviceName": "Pixel 9", "platformName": "Android" }, "success": 1, "report": "", "options": {}, "flows": [ { "id": 123, "name": "login_flow", "report": "...", "requested_at": "2025-11-28T09:15:00.000Z", "completed_at": "2025-11-28T09:16:00.000Z", "status": "DONE", "test_case_id": 456 } ] } ], "success": true, "completed": true } ``` ### Run Status Values | Status | Description | | --- | --- | | `WAITING` | Run is queued and waiting for a device | | `READY` | The test is running on the device | | `DONE` | Run has completed | | `FAILED` | Run failed to complete | | `CANCELLED` | Run was [cancelled](https://testingbot.com#cancel) before it finished, either through the API or from the dashboard | ## Get Run Info Retrieve detailed information about a specific Maestro run. Use this to poll for completion status. | Method | Endpoint | | --- | --- | | `GET` | `/app-automate/maestro/:project_id/:run_id` | [cURL](https://testingbot.com#) ```bash curl -u api_key:api_secret \ "https://api.testingbot.com/v1/app-automate/maestro/:project_id/:run_id" ``` Replace `:project_id` with your project ID and `:run_id` with the run ID. ### Response ```json { "id": 18809, "status": "DONE", "capabilities": { "version": "16", "deviceName": "Pixel 9", "platformName": "Android" }, "flows": [ { "id": 123, "name": "login_flow", "report": "...", "requested_at": "2025-11-28T09:15:00.000Z", "completed_at": "2025-11-28T09:16:00.000Z", "status": "DONE", "test_case_id": 456 } ], "success": 1, "report": "", "options": {}, "completed": true } ``` Poll this endpoint to check if the test has finished. The `completed` field will be `true` when the run has completed. ## JUnit Report Retrieve the JUnit XML report for a completed Maestro run. This is useful for CI/CD integration. | Method | Endpoint | | --- | --- | | `GET` | `/app-automate/maestro/:project_id/:run_id/junit_report` | [cURL](https://testingbot.com#) ```bash curl -u api_key:api_secret \ "https://api.testingbot.com/v1/app-automate/maestro/:project_id/:run_id/junit_report" ``` ### Response ```json { "junit_report": "..." } ``` The `junit_report` field contains the XML-formatted JUnit report generated by Maestro. ## Cancel Maestro Run Immediately cancel a queued or running Maestro run. Cancelling stops the run from requesting any more devices, closes out every flow that has not finished yet, releases the device session the run is currently holding, and drops any request the run still has queued on the device grid, so you stop being billed for it right away. Flows that already completed keep their results and reports. | Method | Endpoint | | --- | --- | | `POST` | `/app-automate/maestro/:project_id/:run_id/cancel` | ### Parameters | Name | Type | Required | Description | | --- | --- | --- | --- | | `project_id` | integer | Yes | The project ID | | `run_id` | integer | Yes | The ID of the run to cancel | [cURL](https://testingbot.com#) ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/maestro/:project_id/:run_id/cancel" ``` ### Response ```json { "success": true, "run": { "id": 18809, "project_id": 4211, "name": "nightly-regression", "status": "CANCELLED", "success": 0, "capabilities": { "platformName": "Android", "version": "16", "deviceName": "Pixel 9" }, "created_at": "2025-11-28T09:15:00.000Z", "completed_at": "2025-11-28T09:15:42.000Z" }, "cancelled_flow_count": 3 } ``` `cancelled_flow_count` is how many flows were still queued or running when the cancel landed. A `0` means the run held no unfinished flows; its device session, if it had one, is still released. ### Error Responses | Status Code | Description | | --- | --- | | `403` | Your account is read-only and cannot start or stop tests. | | `404` | No project or run with those IDs exists on your account. | | `409` | The run had already reached a terminal state (`DONE`, `FAILED` or `CANCELLED`), so there was nothing to cancel. | A cancelled run cannot be resumed, and its individual flows cannot be retried with [Retry Maestro Flow](https://testingbot.com#retry). Use [Retry Maestro Run](https://testingbot.com#retryRun) to start the whole run again on the same capabilities. ## Stop Maestro Run Alias of [Cancel Maestro Run](https://testingbot.com#cancel), kept for existing integrations. It does exactly the same thing, with one difference: it always answers `200`, so stopping a run that has already finished is a no-op rather than a `409`. It returns only a success flag, not the run. New integrations should use the cancel endpoint. | Method | Endpoint | | --- | --- | | `POST` | `/app-automate/maestro/:project_id/:run_id/stop` | [cURL](https://testingbot.com#) ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/maestro/:project_id/:run_id/stop" ``` ### Response ```json { "success": true } ``` ## Retry Maestro Flow Re-run a single flow from a Maestro run. This creates a new flow attempt inside the same run and schedules it on the same device configuration, which is useful for retrying a flaky or failed flow without starting a whole new run. | Method | Endpoint | | --- | --- | | `POST` | `/app-automate/maestro/:project_id/:run_id/:flow_id/retry` | ### Parameters | Name | Type | Required | Description | | --- | --- | --- | --- | | `project_id` | integer | Yes | The project ID | | `run_id` | integer | Yes | The run ID | | `flow_id` | integer | Yes | The ID of the flow (from the run's `flows` array) to retry | [cURL](https://testingbot.com#) ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/maestro/:project_id/:run_id/:flow_id/retry" ``` ### Response ```json { "success": true, "flow": { "id": 124, "name": "login_flow", "status": "WAITING", "requested_at": "2025-11-28T09:20:00.000Z" } } ``` The retry runs as a new flow attempt inside the same run. Poll the [Get Run Info](https://testingbot.com#runInfo) endpoint to track the new flow's status via its returned `id`. ## Retry Maestro Run Re-run an entire Maestro run. This creates a brand new run on the same device configuration (capabilities) and options as the original, re-scheduling every flow. Use this when you want to repeat a full run rather than a single [flow](https://testingbot.com#retry). | Method | Endpoint | | --- | --- | | `POST` | `/app-automate/maestro/:project_id/:run_id/retry` | ### Parameters | Name | Type | Required | Description | | --- | --- | --- | --- | | `project_id` | integer | Yes | The project ID | | `run_id` | integer | Yes | The ID of the run to retry | [cURL](https://testingbot.com#) ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/maestro/:project_id/:run_id/retry" ``` ### Response ```json { "success": true, "run": { "id": 18920, "capabilities": { "platformName": "Android", "version": "14", "deviceName": "Pixel 8" }, "flows": [ { "id": 125, "name": "login_flow", "status": "WAITING", "requested_at": "2025-11-28T09:25:00.000Z" } ] } } ``` A new run is created with its own `id`. Poll the [Get Run Info](https://testingbot.com#runInfo) endpoint with that ID to track its status. ## Frequently asked questions Do I have to use the REST API or can I just use the TestingBot CLI? For most workflows the [TestingBot CLI](https://testingbot.com/support/app-automate/maestro) is simpler since it handles upload, run and result polling in one command. The REST API is the right choice when you want fine-grained control, are integrating from a non-Node environment, or need to fetch artifacts asynchronously. How do I upload my app and Maestro flows via the API? POST your binary to the `/maestro/app` endpoint and your flow ZIP to `/maestro/:id/tests` with HTTP basic auth (API key and secret). The response returns IDs you use when starting the run. See the [Upload App](https://testingbot.com#uploadApp) and [Upload Maestro Flows](https://testingbot.com#uploadFlows) sections. How do I start a Maestro test run via the API? POST to the run endpoint with the app and flow IDs from the upload step, plus your device target and any options. See the [Run Maestro Tests section](https://testingbot.com#run) for the request body and response shape. How do I fetch JUnit-style reports for CI? GET the run's JUnit endpoint to receive a JUnit XML file with per-flow pass and fail results. Most CI runners can consume this format directly. See the [JUnit Report section](https://testingbot.com#report). How do I find a run by its build name? GET `/maestro/runs/:build_name` to resolve a build name straight to its project ID and run ID(s) in one request, instead of listing projects and scanning each one. A build can map to multiple runs (one per device), so the response is always an array. See the [Find Run by Build Name section](https://testingbot.com#findRun). How do I cancel a running Maestro test? POST to the cancel endpoint with the project ID and run ID. The run stops requesting devices, its unfinished flows are closed out, and the device session it holds is released, so billing stops right away. Flows that already finished keep their results. See the [Cancel Maestro Run section](https://testingbot.com#cancel) for details. Can I retry a single flow from a cancelled run? No. A cancelled run cannot be reopened, so flow-level retry returns a `409` for it. Use [Retry Maestro Run](https://testingbot.com#retryRun) to run the whole thing again on the same capabilities. How do I retry a single failed flow? POST to the retry endpoint with the project ID, run ID and flow ID. This re-runs just that flow inside the same run instead of starting a whole new run. See the [Retry Maestro Flow section](https://testingbot.com#retry). How do I retry an entire run? POST to the run-level retry endpoint with the project ID and run ID. This creates a new run on the same capabilities and options, re-scheduling every flow. See the [Retry Maestro Run section](https://testingbot.com#retryRun). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/maestro/options # Maestro Test Options When running Maestro tests on TestingBot, you can customize the test execution using various options. These options are passed when [starting a test run](https://testingbot.com/support/app-automate/maestro/api#run) via the API. Options are organized into two categories: - **maestroOptions** : Options specific to Maestro execution (version, environment variables, flows, tags) - **capabilities** : Device and test configuration options (device, orientation, locale, network, etc.) Moving an existing suite across? See [migrate from Maestro Cloud](https://testingbot.com/support/app-automate/maestro/migrate-from-maestro-cloud) for the full flag mapping. ## Quick Reference All available options at a glance: ### Maestro Options | Option | Type | Description | | --- | --- | --- | | `version` | string | Maestro CLI version to use (e.g., "2.0.10") | | `flows` | array | Only run these flows, by path, file name or directory (e.g., ["login.yml", "checkout.yml"]) | | `env` | object | Environment variables to pass to flows | | `includeTags` | array | Only run flows carrying at least one of these tags. Untagged flows are skipped | | `excludeTags` | array | Skip flows carrying any of these tags. Untagged flows are kept | | `shardSplit` | number | Split flows across N parallel sessions | | `showKeyboard` | boolean | Show the soft keyboard on Android (default: false) | | `commitSha` | string | Git commit SHA for CI/CD tracking | | `pullRequestId` | string | Pull request ID for CI/CD tracking | | `repoName` | string | Repository name (e.g., GitHub repo slug) | | `repoOwner` | string | Repository owner (e.g., GitHub org or username) | ### Capability Options | Option | Type | Description | | --- | --- | --- | | `platformName` | string | Platform: "Android" or "iOS" (required) | | `version` | string | OS version (required) | | `deviceName` | string | Device name (required) | | `name` | string | Test name shown in dashboard | | `build` | string | Build name for grouping tests | | `groups` | comma-separated strings | Groups to which the test belongs, visible in the dashboard | | `orientation` | string | "PORTRAIT" (default) or "LANDSCAPE" | | `locale` | string | Device locale (e.g., "en\_US", "de\_DE") | | `timeZone` | string | Device timezone (e.g., "America/New\_York") | | `googlePlayStore` | boolean | Use Google Play Store Android emulator. | | `throttle_network` | string | Network preset: "Edge", "3G", "4G", "airplane" | | `testingbot.geoCountryCode` | string | Route traffic through specific country (ISO code) | ## Set Maestro Version Unless you pin one, TestingBot runs your flows with Maestro **2.6.0**. This is a known-good default rather than the newest release, so pin a version explicitly if you need a specific one. These versions can be pinned: - `2.10.0`, `2.9.0`, `2.8.0`, `2.7.0` - `2.6.1`, `2.6.0` (default), `2.5.0`, `2.4.0` - `2.3.0`, `2.0.10` **Startup time.** A few versions are pre-installed on our devices and start immediately. Any other version is fetched when the session starts, which adds to your startup time. If a run is slow to start after pinning a version, that is why. [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash testingbot maestro app.apk ./flows \ --device "Pixel 8" \ --deviceVersion "14" \ --maestro-version "2.0.10" ``` If you need a specific version for compatibility, specify the `version` option in `maestroOptions`. | Option | Type | Example | | --- | --- | --- | | `maestroOptions.version` | string | "2.0.10" | ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \ -d '{ "maestroOptions": { "version": "2.0.10" }, "capabilities": [{ "platformName": "Android", "version": "14", "deviceName": "Pixel 8" }] }' \ -H "Content-Type: application/json" ``` Replace `:id` with your project ID from the [app upload](https://testingbot.com/support/app-automate/maestro/api#uploadApp). ## Select Specific Flows By default, TestingBot runs every flow file in your zip archive, one device session per flow. To run only some of them, use the option below. Flows you filter out are not scheduled at all, so they do not consume a session. Each entry is matched against the path of the flow inside your archive, and can be: - the full path, for example `flows/smoke/login.yml` - just the file name, for example `login.yml` - a directory, which selects every flow inside it, for example `flows/smoke` ### Use Cases - Run a subset of flows for faster feedback during development - Execute specific test scenarios without modifying your test suite - Re-run failed flows without running the entire suite [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) With the CLI, you specify flows directly as arguments: ```bash # Run specific flow files testingbot maestro app.apk login.yml checkout.yml --device "Pixel 8" # Run flows from a specific directory testingbot maestro app.apk ./flows/smoke --device "Pixel 8" # Mix directories and individual files testingbot maestro app.apk ./flows/smoke login.yml --device "Pixel 8" ``` | Option | Type | Example | | --- | --- | --- | | `maestroOptions.flows` | array of strings | `["login.yml", "checkout.yml"]` | ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \ -d '{ "maestroOptions": { "flows": ["login.yml", "checkout.yml"] }, "capabilities": [{ "platformName": "Android", "version": "14", "deviceName": "Pixel 8" }] }' \ -H "Content-Type: application/json" ``` **A filter that matches nothing is rejected.** If `flows`, `includeTags` or `excludeTags` leave no flows to run, the run request comes back with a `400` listing the flows the project does have, and nothing is scheduled. Check the paths and tags against the archive you uploaded. ## Environment Variables Pass environment variables to your Maestro flows using the `env` option. This is useful for configuration that varies between environments (staging, production) or for sensitive data like API keys. ### Use Cases - **Environment-specific URLs** : Point to staging or production backends - **Test credentials** : Pass login credentials securely via CI/CD - **Feature flags** : Enable or disable features during tests - **App identifiers** : Specify package names or bundle IDs dynamically [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) Use the `-e` or `--env` flag (can be repeated): ```bash testingbot maestro app.apk ./flows --device "Pixel 8" \ -e API_URL=https://staging.example.com \ -e TEST_USER=testuser@example.com \ -e TEST_PASSWORD=secret123 ``` | Option | Type | Example | | --- | --- | --- | | `maestroOptions.env` | object | `{"API_URL": "https://staging.example.com"}` | ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \ -d '{ "maestroOptions": { "env": { "API_URL": "https://staging.example.com", "TEST_USER": "testuser@example.com", "TEST_PASSWORD": "secret123" } }, "capabilities": [{ "platformName": "Android", "version": "14", "deviceName": "Pixel 8" }] }' \ -H "Content-Type: application/json" ``` ### Using Variables in Your Flows Reference environment variables in your Maestro flow files using the `${VARIABLE_NAME}` syntax: ```yaml appId: com.example.app --- - launchApp - inputText: text: "${TEST_USER}" - tapOn: "Next" - inputText: text: "${TEST_PASSWORD}" - tapOn: "Login" ``` **Best Practice:** Never commit sensitive values like passwords or API keys to your repository. Pass them as environment variables through your CI/CD pipeline. ## Include Tags Run only flows that have specific tags. This is equivalent to Maestro's `--include-tags` option. **Untagged flows are skipped.** When you set `includeTags`, a flow has to carry at least one of the tags you listed to run. Flows with no tags at all are left out, which is how Maestro's own `--include-tags` behaves. Tags are matched without regard to case. ### Adding Tags to Flows Add tags to your flow files using the `tags` property: ```yaml appId: com.example.app tags: - smoke - login --- - launchApp - tapOn: "Login" ``` [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) Use the `--include-tags` flag with comma-separated tags: ```bash testingbot maestro app.apk ./flows --device "Pixel 8" \ --include-tags "smoke,critical" ``` | Option | Type | Example | | --- | --- | --- | | `maestroOptions.includeTags` | array of strings | `["smoke", "critical"]` | ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \ -d '{ "maestroOptions": { "includeTags": ["smoke", "critical"] }, "capabilities": [{ "platformName": "Android", "version": "14", "deviceName": "Pixel 8" }] }' \ -H "Content-Type: application/json" ``` ## Exclude Tags Skip flows that have specific tags. This is equivalent to Maestro's `--exclude-tags` option. Unlike [includeTags](https://testingbot.com#include-tags), flows with no tags are kept. You can combine the two. `includeTags` is applied first and `excludeTags` second, so a flow tagged both `smoke` and `flaky` is skipped by `"includeTags": ["smoke"], "excludeTags": ["flaky"]`. ### Use Cases - Skip slow tests during development for faster feedback - Exclude known flaky tests from CI pipelines - Skip platform-specific tests (e.g., exclude "ios-only" when running on Android) [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) Use the `--exclude-tags` flag with comma-separated tags: ```bash testingbot maestro app.apk ./flows --device "Pixel 8" \ --exclude-tags "slow,flaky" ``` | Option | Type | Example | | --- | --- | --- | | `maestroOptions.excludeTags` | array of strings | `["slow", "flaky"]` | ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \ -d '{ "maestroOptions": { "excludeTags": ["slow", "flaky"] }, "capabilities": [{ "platformName": "Android", "version": "14", "deviceName": "Pixel 8" }] }' \ -H "Content-Type: application/json" ``` ## IP Geolocation Test your app from different geographic locations using TestingBot's GeoIP feature. Traffic from the device will route through an IP address in the specified country. ### Use Cases - Test geo-restricted content or features - Verify location-based pricing or currency display - Test localized content delivery [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash testingbot maestro app.zip ./flows \ --platform iOS \ --device "iPhone 15" \ --deviceVersion "17.2" \ --geo-country-code "DE" ``` | Option | Type | Example | | --- | --- | --- | | `testingbot.geoCountryCode` | string (ISO code) | `"DE", "US", "JP"` | ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \ -d '{ "capabilities": [{ "platformName": "iOS", "version": "17.2", "deviceName": "iPhone 15", "testingbot.geoCountryCode": "DE" }] }' \ -H "Content-Type: application/json" ``` See the full list of [available country codes](https://testingbot.com/support/web-automate/selenium/test-options#geo). ## Network Throttling Simulate different network conditions to test how your app performs under various connectivity scenarios. | Option | Type | Example | | --- | --- | --- | | `throttle_network` | string | "4G", "3G", "Edge", "airplane" | ### Network Presets | Preset | Download | Upload | Latency | Use Case | | --- | --- | --- | --- | --- | | `4G` | 18 Mbps | 9 Mbps | 100ms | Standard mobile connection | | `3G` | 400 Kbps | 100 Kbps | 100ms | Slower mobile network | | `Edge` | 250 Kbps | 150 Kbps | 300ms | Poor connectivity areas | | `airplane` | 0 | 0 | - | No connectivity (offline mode) | | `disable` | - | - | - | Restore full connectivity | [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash testingbot maestro app.zip ./flows \ --platform iOS \ --device "iPhone 15" \ --deviceVersion "17.2" \ --throttle-network "3G" ``` ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \ -d '{ "capabilities": [{ "platformName": "iOS", "version": "17.2", "deviceName": "iPhone 15", "throttle_network": "3G" }] }' \ -H "Content-Type: application/json" ``` ## Test Name Set a custom name for your test that appears in the TestingBot dashboard. This makes it easier to identify specific test runs. | Option | Type | Example | | --- | --- | --- | | `name` | string | "Login Flow - Smoke Test" | [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash testingbot maestro app.zip ./flows \ --platform iOS \ --device "iPhone 15" \ --deviceVersion "17.2" \ --name "Login Flow - Smoke Test" ``` ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \ -d '{ "capabilities": [{ "platformName": "iOS", "version": "17.2", "deviceName": "iPhone 15", "name": "Login Flow - Smoke Test" }] }' \ -H "Content-Type: application/json" ``` ## Build Grouping Group multiple test runs under a single build name. This is useful for organizing tests that belong to the same CI/CD pipeline run or release version. | Option | Type | Example | | --- | --- | --- | | `build` | string | "Release 2.5.0" | ### Benefits - View aggregated pass/fail statistics for all tests in a build - Track test results across different device configurations - Easily identify which builds introduced failures [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash # The Maestro CLI groups runs by the --name you pass; the API exposes this as the "build" field testingbot maestro app.ipa ./flows \ --platform iOS \ --device "iPhone 17" \ --real-device \ --name "Release 2.5.0" ``` ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \ -d '{ "capabilities": [{ "platformName": "iOS", "version": "26.0", "deviceName": "iPhone 17", "realDevice": true, "build": "Release 2.5.0" }] }' \ -H "Content-Type: application/json" ``` ## Shard Split Group your Maestro flows into chunks, which will run across multiple parallel sessions for faster execution. For example you have 3 parallel slots and 9 tests, but you want to split this test suite into 3 chunks of tests and run them in parallel on connected devices, then you would use `shardSplit` set to 3. | Option | Type | Example | | --- | --- | --- | | `shardSplit` | number | 1 | [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash # Split flows across 3 parallel sessions - without this each flow would run in its own session testingbot maestro app.apk ./flows \ --device "Pixel 8" \ --deviceVersion "14" \ --shard-split 3 ``` ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \ -d '{ "shardSplit": 3, "capabilities": [{ "platformName": "Android", "version": "14", "deviceName": "Pixel 8" }] }' \ -H "Content-Type: application/json" ``` ## Device Matrix Run the same flows across several devices in one command with `--device-matrix`. Each cell names exactly one device as `[:][:real]`; cells are comma-separated or the flag is repeated. There is no cross-product, because not every device exists in every OS version. ```bash testingbot maestro app.apk ./flows \ --device-matrix "Pixel 9:14" \ --device-matrix "Samsung Galaxy S24:14:real" \ --device-matrix "Pixel 8" ``` Every flow runs once per device, so the cost is devices × flows; the CLI prints that summary before submitting. Each device becomes its own run with its own results, live table rows and dashboard link, `--json` lists the device per run, and `--retry` re-runs only the flow that failed on the device it failed on. If any cell is not a valid device and OS combination the whole request is rejected and nothing runs, so a matrix never partially submits. `--real-device`, or an `.ipa` app which implies it, applies to every cell. `--device` and `--deviceVersion` cannot be combined with a matrix; list every device as a cell instead. ## Reuse an Uploaded App Every run prints its project ID after the app upload. Pass that ID to a later run with `--app-binary-id` and the upload is skipped; each run still gets its own project and results. `testingbot upload` uploads an app on its own and prints the ID, which is handy for pipelines that build once and run several suites. ```bash testingbot upload app.apk # App ready: app.apk. Project ID: 4321 # Run flows against it with: testingbot maestro --app-binary-id 4321 ./flows testingbot maestro --app-binary-id 4321 ./flows/smoke testingbot maestro --app-binary-id 4321 ./flows/regression --device "Pixel 9" ``` The platform is taken from the stored app unless `--platform` is given. Unchanged binaries are also deduplicated by checksum on every upload; pass `--ignore-checksum-check` to force a fresh upload. ## Expo / EAS Build `eas build` hands back a download URL rather than a local file, and for iOS simulator builds the artifact is a `.tar.gz` that contains the `.app`. Pass the URL with `--app-url` and the CLI downloads it, extracts the bundle when needed, and uploads it like any other app. Android EAS builds (`.apk`) and plain `.zip` / `.ipa` URLs work the same way, and a local `.tar.gz` can be passed as the app file directly. ```bash # iOS simulator build URL=$(eas build --platform ios --profile preview --json --non-interactive | jq -r '.[0].artifacts.buildUrl') testingbot maestro --app-url "$URL" ./flows --device "iPhone 16" # Android build URL=$(eas build --platform android --profile preview --json --non-interactive | jq -r '.[0].artifacts.buildUrl') testingbot maestro --app-url "$URL" ./flows --device "Pixel 9" ``` Inside an EAS Build or EAS Workflows job the run is tagged automatically with the EAS build id, profile and platform, and the job's commit hash is used as the commit SHA unless `--commit-sha` is given. EAS download links are signed and expire after about an hour; a rejected link is reported as such so you know to request a fresh one. `testingbot upload ` accepts URLs too, which is handy to [upload once and reuse](https://testingbot.com#app-binary-id) across several suites. ## JSON Output and Exit Codes `--json` prints one JSON document on stdout with the outcome, project ID, dashboard URL and every flow attempt per run, and moves all log lines to stderr. `--json-file` writes the same document to `_testingbot.json` (or `--json-file-name`) while keeping the console output. Exit codes are `0` passed, `2` flows failed, `1` CLI or infrastructure error. Examples and the document layout are on the [CI/CD page](https://testingbot.com/support/app-automate/maestro/ci-cd#json). ## CI/CD Integration Pass Git and repository metadata to associate test runs with specific commits and pull requests. This information appears in the TestingBot dashboard and can be used for tracking and debugging. | Option | Type | Description | | --- | --- | --- | | `commitSha` | string | Git commit SHA associated with this test run | | `pullRequestId` | string | Pull request ID this test run originated from | | `repoName` | string | Repository name (e.g., GitHub repo slug) | | `repoOwner` | string | Repository owner (e.g., GitHub organization or username) | [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash # Pass Git metadata for CI/CD tracking testingbot maestro app.apk ./flows \ --device "Pixel 8" \ --deviceVersion "14" \ --commit-sha "abc123def456" \ --pull-request-id "42" \ --repo-owner "myorg" \ --repo-name "myapp" ``` ### GitHub Actions Example The [TestingBot Maestro GitHub Action](https://github.com/testingbot/maestro-action) fills in the branch, commit, repository and pull request from the workflow context, so no metadata flags are needed: ```yaml - uses: testingbot/maestro-action@v1 with: api-key: ${{ secrets.TB_KEY }} api-secret: ${{ secrets.TB_SECRET }} app-file: app.apk workspace: ./flows device: Pixel 8 ``` See [GitHub Actions](https://testingbot.com/support/app-automate/maestro/ci-cd/github-actions) for the full workflow and the action's outputs. ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \ -d '{ "metadata": { "commitSha": "abc123def456", "pullRequestId": "42", "repoName": "myapp", "repoOwner": "myorg" }, "capabilities": [{ "platformName": "Android", "version": "14", "deviceName": "Pixel 8" }] }' \ -H "Content-Type: application/json" ``` ## Device Orientation Set the device orientation at the start of your test. | Option | Type | Values | Default | | --- | --- | --- | --- | | `orientation` | string | `"PORTRAIT", "LANDSCAPE"` | PORTRAIT | [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash testingbot maestro app.apk ./flows \ --device "Pixel 8" \ --deviceVersion "14" \ --orientation LANDSCAPE ``` ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \ -d '{ "capabilities": [{ "platformName": "Android", "version": "14", "deviceName": "Pixel 8", "orientation": "LANDSCAPE" }] }' \ -H "Content-Type: application/json" ``` ## Soft Keyboard (Android) By default, the Android soft keyboard is hidden during Maestro tests. Use the `showKeyboard` option to display the soft keyboard on Android devices. | Option | Type | Default | Platform | | --- | --- | --- | --- | | `maestroOptions.showKeyboard` | boolean | false | Android only | ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \ -d '{ "maestroOptions": { "showKeyboard": true }, "capabilities": [{ "platformName": "Android", "version": "14", "deviceName": "Pixel 8" }] }' \ -H "Content-Type: application/json" ``` ## Platform Configuration Platform-specific settings allow you to optimize the environment for Android or iOS. These are configured in your [config.yaml](https://testingbot.com#config) file under the `platform` key. ### iOS specifics | Key | Description | | --- | --- | | `disableAnimations` | Enables Reduce Motion on the iOS Simulator to prevent flakiness caused by system-level animations. | ### Android specifics | Key | Description | | --- | --- | | `disableAnimations` | Disables system-level window, transition and animator animations on the Android Emulator. | ### Example config.yaml ```yaml platform: ios: disableAnimations: true android: disableAnimations: true ``` ## Device Timezone Set the device timezone for your test. Useful for testing time-sensitive features. | Option | Type | Example | | --- | --- | --- | | `timeZone` | string (IANA) | `"America/New_York", "Europe/London", "Asia/Tokyo"` | [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash testingbot maestro app.zip ./flows \ --platform iOS \ --device "iPhone 15" \ --deviceVersion "17.2" \ --timezone "America/New_York" ``` ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \ -d '{ "capabilities": [{ "platformName": "iOS", "version": "17.2", "deviceName": "iPhone 15", "timeZone": "America/New_York" }] }' \ -H "Content-Type: application/json" ``` See the [full list of timezone identifiers](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones). ## Device Locale Set the device locale to test localized content and formatting. The locale affects language, date formats, number formats, and currency display. | Option | Type | Format | Example | | --- | --- | --- | --- | | `locale` | string | language\_COUNTRY | `"de_DE", "fr_CA", "ja_JP"` | [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash testingbot maestro app.zip ./flows \ --platform iOS \ --device "iPhone 15" \ --deviceVersion "17.2" \ --device-locale "de_DE" ``` ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \ -d '{ "capabilities": [{ "platformName": "iOS", "version": "17.2", "deviceName": "iPhone 15", "locale": "de_DE" }] }' \ -H "Content-Type: application/json" ``` ### Common Locale Codes | Region | Locale Code | | --- | --- | | English (US) | `en_US` | | English (UK) | `en_GB` | | German (Germany) | `de_DE` | | French (France) | `fr_FR` | | French (Canada) | `fr_CA` | | Spanish (Spain) | `es_ES` | | Spanish (Mexico) | `es_MX` | | Japanese | `ja_JP` | | Chinese (Simplified) | `zh_CN` | | Chinese (Traditional) | `zh_TW` | | Korean | `ko_KR` | | Portuguese (Brazil) | `pt_BR` | | Italian | `it_IT` | | Dutch | `nl_NL` | | Russian | `ru_RU` | | Arabic | `ar_SA` | | Hindi | `hi_IN` | ## Maestro config.yaml You can include a `config.yaml` file in your [Maestro flows zip file](https://testingbot.com/support/app-automate/maestro#uploadSuite). This configuration file lets you define options that apply to all your flows. ### Supported config.yaml Options | Option | Description | | --- | --- | | `includeTags` | Only run flows with these tags | | `excludeTags` | Skip flows with these tags | | `env` | Environment variables available in all flows | | `platform` | Platform-specific settings for iOS and Android (see [Platform Configuration](https://testingbot.com#platform-configuration)) | ### Example config.yaml ```yaml env: API_URL: https://api.example.com DEBUG: "true" includeTags: - smoke - critical ``` ## TestingBot Tunnel Use [TestingBot Tunnel](https://testingbot.com/support/tunnel) to allow your app to access private or staging backend servers during tests. ### Use Cases - Test against staging APIs not accessible from the public internet - Access localhost services running on your machine - Test with private databases or internal services When a TestingBot Tunnel is active on your account, Maestro tests automatically route traffic through it. No additional configuration is needed. To start a tunnel, download and run the [TestingBot Tunnel](https://testingbot.com/support/tunnel) client before starting your tests. You can also use the built-in [tunnel](https://testingbot.com#tunnel-options) option to automatically start a tunnel for the duration of your test run. See the dedicated [Maestro Tunnel guide](https://testingbot.com/support/app-automate/maestro/tunnel) for use cases, a staging token helper and CI/CD examples. ## CLI Reference When using the [TestingBot CLI](https://testingbot.com/support/app-automate/maestro#cli), these options are available as command-line flags: ### App & Flows The application and flows are normally passed as positional arguments (`testingbot maestro app.apk ./flows`), but you can also set them explicitly: | CLI Option | Description | | --- | --- | | `--app ` | Path to the application under test (.apk, .ipa, .app, or .zip) | | `--other-app ` | Additional app to install alongside `--app`. Can be repeated, up to 4 times. | | `--app-url ` | Download the app from an http(s) URL instead of a local file: `.apk`, `.ipa`, `.zip` or an EAS Build iOS `.tar.gz` (the `.app` inside is extracted automatically). Every positional argument is then a flow. Signed URLs such as EAS links expire after about an hour. See [Expo / EAS Build](https://testingbot.com#expo). | | `--app-binary-id ` | Reuse the app of a project uploaded earlier (`testingbot upload` or any previous run's project ID) instead of uploading one. Every positional argument is then a flow. See [Reuse an Uploaded App](https://testingbot.com#app-binary-id). | ### Device Options | CLI Option | Description | | --- | --- | | `--device ` | Device name (e.g., "Pixel 9", "iPhone 17") | | `--platform ` | Platform: Android or iOS | | `--deviceVersion ` | OS version (e.g., "14", "17.2") | | `--real-device` | Use a real device instead of emulator/simulator | | `--device-matrix ` | Run every flow on each listed device: `[:][:real]`, comma-separated or repeatable. Cannot be combined with `--device` or `--deviceVersion`. See [Device Matrix](https://testingbot.com#device-matrix). | | `--google-play` | Use the Google Play Store-enabled version (Android emulator only) | | `--orientation ` | Screen orientation: PORTRAIT or LANDSCAPE | | `--device-locale ` | Device locale (e.g., "en\_US", "de\_DE") | | `--timezone ` | Timezone (e.g., "America/New\_York", "Europe/London") | ### Test Configuration | CLI Option | Description | | --- | --- | | `--name ` | Name for this Maestro run, used for dashboard identification and build grouping | | `--groups ` | Tag the test session with one or more groups (comma-separated) | | `--include-tags ` | Only run flows with these tags (comma-separated) | | `--exclude-tags ` | Exclude flows with these tags (comma-separated) | | `--exclude-flows ` | Flow files, directories or glob patterns to leave out of the run (comma-separated, repeatable). An excluded flow that another flow still invokes via `runFlow` is bundled as a subflow but never runs on its own. | | `-e, --env ` | Environment variable for flows (can be repeated) | | `--maestro-version ` | Maestro version to use (e.g., "2.0.10") | | `--config ` | Path to a custom Maestro config file (default: config.yaml in project root) | ### Execution Options | CLI Option | Description | | --- | --- | | `--retry ` | Retry failed flows up to N times (0-2, default 0). Stops as soon as a flow passes. | | `--async` | Start tests and exit immediately without waiting for results | | `--dry-run` | Validate and prepare everything but skip the HTTP calls. Shows what would be sent. | | `-q, --quiet` | Quieter console output without progress updates | | `--json` | Print results as a single JSON document on stdout (logs move to stderr). Implies `--quiet`. Exit code 2 when flows fail. See [JSON Output](https://testingbot.com#json). | | `--json-file` | Write the JSON results to a file (default `_testingbot.json`). Exit code stays 0 when flows fail so the pipeline can gate on the file. | | `--json-file-name ` | Custom path for the JSON results file (requires `--json-file`) | ### CI/CD Integration | CLI Option | Description | | --- | --- | | `--commit-sha ` | Git commit SHA associated with this test run | | `--pull-request-id ` | Pull request ID this test run originated from | | `--repo-name ` | Repository name (e.g., GitHub repo slug) | | `--repo-owner ` | Repository owner (e.g., GitHub organization or username) | | `--branch ` | Git branch this test run was built from | | `--pr-url ` | Pull request URL this test run originated from | | `-m, --metadata ` | Free-form metadata attached to the run and shown in the dashboard (repeatable) | | `--check-name ` | Name the GitHub pull request check `TestingBot / `, so several runs on one commit (iOS, Android) post separate checks. See [GitHub PR Checks](https://testingbot.com/support/app-automate/maestro/ci-cd/github-pr-checks#one-check). | With the [TestingBot GitHub App](https://testingbot.com/support/app-automate/maestro/ci-cd/github-pr-checks) installed, a run carrying the repository and full commit SHA also posts a status check on the pull request. ### Network & Location | CLI Option | Description | | --- | --- | | `--throttle-network ` | Network throttling: 4G, 3G, Edge, airplane, or disable | | `--geo-country-code ` | Geographic IP location (ISO country code, e.g., "US", "DE") | ### Report & Artifacts | CLI Option | Description | | --- | --- | | `--report ` | Download report after completion: html, html-detailed, junit, or allure (Allure result files under `allure-results/`) | | `--report-output-dir ` | Directory to save reports (required when `--report` is used) | | `--download-artifacts [mode]` | Download test artifacts (logs, screenshots, video) after completion. Mode: `all` (default) or `failed`. | | `--artifacts-output-dir ` | Directory to save artifacts zip (defaults to the current directory) | ### Tunnel Options | CLI Option | Description | | --- | --- | | `-t, --tunnel` | Start a [TestingBot Tunnel](https://testingbot.com/support/tunnel) for this test run (cannot be combined with `--async`) | | `--tunnel-identifier ` | Identifier for the tunnel, allowing multiple tunnels in parallel | ### Advanced Options | CLI Option | Description | | --- | --- | | `--ignore-checksum-check` | Skip checksum verification and always upload the app | | `--shard-split ` | Split flows into N parallel sessions for faster execution. More information in the [Shard Splitting](https://testingbot.com#shard-split) section. | ### Authentication & Debugging Credentials are normally resolved from `testingbot login`, the `TB_KEY` / `TB_SECRET` environment variables, or a `~/.testingbot` file. You can also pass them explicitly: | CLI Option | Description | | --- | --- | | `--api-key ` | TestingBot API key | | `--api-secret ` | TestingBot API secret | | `--debug` | Enable debug logging of API responses | ### Other Commands Besides `maestro`, the CLI has commands for working with projects after they were started. All of them accept the authentication and `--json` flags above. | Command | Description | | --- | --- | | `testingbot upload ` | Upload an app once and print a project ID to reuse with `--app-binary-id` | | `testingbot status --id [--wait]` | Show every run and flow of a project; `--wait` blocks with the live flow table until it finishes (Ctrl-C detaches without cancelling) | | `testingbot artifacts --id ` | Download `--report` and/or `--download-artifacts` for a finished project | | `testingbot list [--count] [--offset]` | List recent Maestro projects, newest first | | `testingbot login` | Authenticate via the browser and store the credentials in `~/.testingbot` | See [Async runs, status and artifacts](https://testingbot.com/support/app-automate/maestro/ci-cd#async) for a pipeline example. ## Frequently asked questions How do I pin a specific Maestro version on TestingBot? Pass the `maestroVersion` option in your run configuration. See the [Set Maestro Version section](https://testingbot.com#version) for the list of supported versions and CLI / API examples. How do I pass environment variables to Maestro flows? Use the `env` option to inject environment variables into the Maestro run. The flows can then read them with the `${VARIABLE_NAME}` syntax inside the YAML. See the [Environment Variables section](https://testingbot.com#environment) for examples. Can I run only a subset of my Maestro flows? Yes. Use `flows` to whitelist specific files, or `includeTags` and `excludeTags` to filter by Maestro tag. Only the flows that survive the filter are scheduled, so the rest do not consume a device session, and a filter that matches nothing is rejected rather than run. See the [Select Specific Flows](https://testingbot.com#flows), [Include Tags](https://testingbot.com#include-tags) and [Exclude Tags](https://testingbot.com#exclude-tags) sections. How do I split Maestro flows into parallel shards? Pass `shardSplit` with the number of parallel sessions you want. Each shard runs a subset of the flows on its own device session. See the [Shard Split section](https://testingbot.com#shard-split) for details. Can I change the device timezone, locale or geolocation for Maestro tests? Yes. Use the `timezone`, `locale` and `geolocation` options to control the device environment. See the [Device Timezone](https://testingbot.com#timezone), [Device Locale](https://testingbot.com#locale) and [IP Geolocation](https://testingbot.com#geolocation) sections. Can I test a private or staging server with Maestro on TestingBot? Yes. Start the [TestingBot Tunnel](https://testingbot.com/support/tunnel) and pass `tunnelIdentifier` in your Maestro run config. The device will route traffic through your tunnel so it can reach internal APIs. See the [TestingBot Tunnel section](https://testingbot.com#tunnel). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/maestro/ci-cd # CI/CD Integration The [TestingBot CLI](https://testingbot.com/support/app-automate/maestro#cli) integrates seamlessly with any CI/CD environment. It runs all your Maestro flows, streams the output in real-time and exits with the appropriate exit code for pipeline automation. We recommend using `npx` to run the CLI without installing it globally. This ensures you're always using the latest version. ## Quick Start Run Maestro tests in any CI/CD service with a single command: ```bash npx --yes @testingbot/cli@latest maestro app.apk ./flows \ --device "Pixel 8" \ --deviceVersion "14" \ --api-key $TB_KEY \ --api-secret $TB_SECRET ``` This command will: - Upload your mobile app (`.apk`, `.aab`, `.ipa` or `.app`) - Upload and run all Maestro flow files from the `./flows` directory - Stream test progress to your CI logs - Exit with code `0` on success, non-zero on failure You can pass credentials as arguments (`--api-key` and `--api-secret`) or set them as environment variables. Environment variables are the recommended approach for CI/CD. ## Environment Variables Instead of passing credentials as command arguments, you can set them as environment variables. This is the most secure approach for CI/CD pipelines since credentials are never exposed in logs or command history. | Variable | Description | | --- | --- | | `TB_KEY` | Your TestingBot API key | | `TB_SECRET` | Your TestingBot API secret | When these environment variables are set, you can omit the credential arguments entirely: ```bash export TB_KEY="your_api_key" export TB_SECRET="your_api_secret" npx --yes @testingbot/cli@latest maestro app.apk ./flows \ --device "Pixel 8" \ --deviceVersion "14" ``` Never hardcode credentials in your CI configuration files. Always use your CI provider's secrets management feature to store `TB_KEY` and `TB_SECRET`. ## Exit Codes The CLI distinguishes a red test run from a broken pipeline, so a job can retry on infrastructure errors but fail the build on real test failures: | Exit Code | Description | | --- | --- | | `0` | All flows passed. Also returned for `--async`, `--dry-run`, and for failed flows when `--json-file` is used (the file is the contract then) | | `1` | CLI or infrastructure error: invalid arguments, missing credentials, upload failure, timeout | | `2` | One or more flows failed | `testingbot status` follows the same rule and additionally exits `0` while a project is still running. ## Test Reports Generate test reports for CI/CD result decoration and test analytics. The CLI supports JUnit XML, HTML and Allure report formats, plus a machine-readable JSON summary. ### JUnit Report Generate JUnit XML reports for integration with CI/CD test result visualization: ```bash npx --yes @testingbot/cli@latest maestro app.apk ./flows \ --device "Pixel 8" \ --deviceVersion "14" \ --report junit \ --report-output-dir ./test-results ``` | Option | Description | | --- | --- | | `--report ` | Report format: `junit` | | `--report-output-dir ` | Directory to save test reports (required when `--report` is used) | JUnit reports can be used with most CI/CD platforms for: - Test result visualization in your pipeline - Tracking test trends over time - Annotating pull requests with test results - Integration with test management tools You can also fetch reports via the [REST API](https://testingbot.com/support/app-automate/maestro/api#report) if you need to retrieve them separately. ### Allure Report `--report allure` converts each run's results into Allure result files under `/allure-results/`, one JSON per flow with its steps, timings, status and failure details. Render them with the [Allure CLI](https://allurereport.org/docs/v2/install/); results from several runs or shards accumulate in the same directory. ```bash npx --yes @testingbot/cli@latest maestro app.apk ./flows --report allure --report-output-dir ./reports allure serve ./reports/allure-results ``` ### JSON Output `--json` prints a single JSON document on stdout and moves all log lines to stderr, so it pipes straight into `jq`. `--json-file` writes the same document to `_testingbot.json` (or the path given with `--json-file-name`) while keeping the normal console output. Both imply `--quiet`. ```bash npx --yes @testingbot/cli@latest maestro app.apk ./flows --json | jq '.runs[].flows[] | {name, passed, durationSeconds}' ``` ```json { "provider": "maestro", "outcome": "failed", "success": false, "appId": 1234, "url": "https://testingbot.com/members/maestro/1234", "runs": [ { "id": 5678, "status": "DONE", "passed": false, "device": { "name": "Pixel 9", "platform": "Android", "version": "14" }, "url": "https://testingbot.com/members/maestro/1234/runs/5678", "flows": [ { "id": 1, "runId": 5678, "name": "login", "status": "DONE", "passed": true, "attempt": 1, "latest": true, "durationSeconds": 30, "errors": [] } ] } ] } ``` `outcome` is one of `passed`, `failed`, `started` (async), `dry-run` or `error`. `flows` lists every attempt, including `--retry` re-runs; `latest` marks the attempt whose verdict counts. When the CLI fails before a verdict, the document still arrives with `outcome: "error"` and an `error` message, so a consumer never has to handle missing output. ## CI/CD Metadata Pass repository and commit information to link test runs with your source control. This enables better traceability and integration with your development workflow. | Option | Description | | --- | --- | | `--commit-sha ` | The commit SHA of this upload | | `--pull-request-id ` | The ID of the pull request this upload originated from | | `--repo-name ` | Repository name (e.g., GitHub repo slug) | | `--repo-owner ` | Repository owner (e.g., GitHub organization or user) | | `--branch ` | Git branch this run was built from | | `--pr-url ` | URL of the pull request this run originated from | | `-m, --metadata ` | Free-form metadata attached to the run (repeatable), for example `-m team=mobile -m env=staging` | Example with metadata: ```bash npx --yes @testingbot/cli@latest maestro app.apk ./flows \ --device "Pixel 8" \ --deviceVersion "14" \ --commit-sha "$COMMIT_SHA" \ --branch "$BRANCH" \ --pull-request-id "$PR_NUMBER" \ --repo-owner "your-org" \ --repo-name "your-repo" \ -m team=mobile ``` With the [TestingBot GitHub App](https://testingbot.com/support/app-automate/maestro/ci-cd/github-pr-checks) installed, a run that carries the repository and the full head commit SHA also posts a `TestingBot / tests` status check on the pull request, so you can require green Maestro flows before merging. No extra step is needed. ## Async runs, status and artifacts Long suites do not have to hold a CI runner. Start the run with `--async`, capture the project ID, and come back for the result in a later job or from a different machine. ```bash # Start and capture the project id PROJECT_ID=$(npx --yes @testingbot/cli@latest maestro app.apk ./flows --async --json | jq -r .appId) # Later: block until it finishes (exit 2 if any flow failed), with the live flow table npx --yes @testingbot/cli@latest status --id "$PROJECT_ID" --wait # Fetch reports and artifacts once it finished npx --yes @testingbot/cli@latest artifacts --id "$PROJECT_ID" --report junit --report-output-dir ./reports npx --yes @testingbot/cli@latest artifacts --id "$PROJECT_ID" --download-artifacts failed --artifacts-output-dir ./artifacts # Browse recent projects npx --yes @testingbot/cli@latest list --count 20 ``` `status --wait` only watches the project; interrupting it does not cancel the run. `status` without `--wait` prints a snapshot and exits `0` while the project is still running, so it also works for polling. ### Expo / EAS Build Pass an `eas build` download URL with `--app-url`, or run inside EAS Workflows with the `@testingbot/eas-workflow` wrapper. Both are covered on the [Expo / EAS page](https://testingbot.com/support/app-automate/maestro/ci-cd/expo-eas). ### Upload the app once, run many suites Every run prints its project ID after the app upload. Pass it to a later run with `--app-binary-id` to skip the upload entirely; each run still gets its own project and results. `testingbot upload` does the upload on its own and prints the ID. ```bash APP_ID=$(npx --yes @testingbot/cli@latest upload app.apk --json | jq -r .appId) npx --yes @testingbot/cli@latest maestro --app-binary-id "$APP_ID" ./flows/smoke --device "Pixel 9" npx --yes @testingbot/cli@latest maestro --app-binary-id "$APP_ID" ./flows/regression --device-matrix "Pixel 9:14,Samsung Galaxy S24:14:real" ``` ## View Result Details A link to the current test results will be printed to your CI logs during execution. The test details page provides detailed test result information, including: - Real-time test execution status - Video recordings of each test run - Screenshots captured during tests - Device logs and Maestro output - Test artifacts for debugging ## GitHub Actions The [TestingBot Maestro GitHub Action](https://testingbot.com/support/app-automate/maestro/ci-cd/github-actions) wraps the CLI, fills in branch, commit, repository and pull request from the workflow context, writes a results table to the job summary and exposes the results as outputs: ```yaml - uses: testingbot/maestro-action@v1 with: api-key: ${{ secrets.TB_KEY }} api-secret: ${{ secrets.TB_SECRET }} app-file: app/build/outputs/apk/debug/app-debug.apk workspace: .maestro ``` Inputs, outputs, a device-matrix example and the raw-CLI alternative are on the [GitHub Actions page](https://testingbot.com/support/app-automate/maestro/ci-cd/github-actions). ## GitHub PR status checks With the TestingBot GitHub App installed, every Maestro run started from a pull request posts a `TestingBot / tests` check listing each flow on each device, and can be made a required check so a pull request cannot be merged while a flow is failing. Nothing to configure beyond installing the app. See [GitHub PR Checks for Maestro](https://testingbot.com/support/app-automate/maestro/ci-cd/github-pr-checks). ## GitLab CI Add Maestro testing to your `.gitlab-ci.yml`. Store your credentials in **Settings → CI/CD → Variables** and mark them as protected and masked. ```yaml stages: - build - test build: stage: build image: gradle:8-jdk17 script: - ./gradlew assembleDebug artifacts: paths: - app/build/outputs/apk/debug/ maestro-tests: stage: test image: node:20 dependencies: - build script: - npx --yes @testingbot/cli@latest maestro app/build/outputs/apk/debug/app-debug.apk ./maestro-flows --device "Pixel 8" --deviceVersion "14" --report junit --report-output-dir ./test-results --commit-sha $CI_COMMIT_SHA --pull-request-id $CI_MERGE_REQUEST_IID --repo-owner $CI_PROJECT_NAMESPACE --repo-name $CI_PROJECT_NAME variables: TB_KEY: $TB_KEY TB_SECRET: $TB_SECRET artifacts: when: always paths: - test-results/ reports: junit: test-results/*.xml ``` Add `TB_KEY` and `TB_SECRET` as CI/CD variables in your GitLab project settings. Enable "Mask variable" to prevent them from appearing in job logs. ## Jenkins Add Maestro testing to your Jenkins pipeline (`Jenkinsfile`). Store your credentials using **Manage Jenkins → Manage Credentials** as secret text. ```groovy pipeline { agent any environment { TB_KEY = credentials('testingbot-api-key') TB_SECRET = credentials('testingbot-api-secret') } stages { stage('Build') { steps { sh './gradlew assembleDebug' } } stage('Maestro Tests') { steps { sh """ npx --yes @testingbot/cli@latest maestro \\ app/build/outputs/apk/debug/app-debug.apk \\ ./maestro-flows \\ --device "Pixel 8" \\ --deviceVersion "14" \\ --report junit \\ --report-output-dir test-results \\ --commit-sha ${env.GIT_COMMIT} \\ --pull-request-id ${env.CHANGE_ID ?: ''} \\ --repo-owner "your-org" \\ --repo-name "your-repo" """ } post { always { junit 'test-results/*.xml' archiveArtifacts artifacts: 'test-results/**', allowEmptyArchive: true } } } } } ``` Create credentials of type "Secret text" in Jenkins for `TB_KEY` and `TB_SECRET`. Reference them using the `credentials()` helper in your pipeline. ## CircleCI Add Maestro testing to your `.circleci/config.yml`. Store your credentials in **Project Settings → Environment Variables**. ```yaml version: 2.1 jobs: maestro-tests: docker: - image: cimg/node:20 steps: - checkout - run: name: Run Maestro Tests command: | npx --yes @testingbot/cli@latest maestro \ ./app.apk \ ./maestro-flows \ --device "Pixel 8" \ --deviceVersion "14" \ --report junit \ --report-output-dir test-results \ --commit-sha $CIRCLE_SHA1 \ --pull-request-id $CIRCLE_PR_NUMBER \ --repo-owner $CIRCLE_PROJECT_USERNAME \ --repo-name $CIRCLE_PROJECT_REPONAME - store_test_results: path: test-results - store_artifacts: path: test-results destination: maestro-results workflows: test: jobs: - maestro-tests ``` Add `TB_KEY` and `TB_SECRET` as environment variables in your CircleCI project settings. CircleCI automatically masks these values in build output. ## Bitbucket Pipelines Add Maestro testing to your `bitbucket-pipelines.yml`. Store your credentials in **Repository settings → Pipelines → Repository variables** and enable "Secured". ```yaml image: node:20 pipelines: default: - step: name: Build and Test caches: - node script: - ./gradlew assembleDebug - npx --yes @testingbot/cli@latest maestro app/build/outputs/apk/debug/app-debug.apk ./maestro-flows --device "Pixel 8" --deviceVersion "14" --report junit --report-output-dir test-results --commit-sha $BITBUCKET_COMMIT --pull-request-id $BITBUCKET_PR_ID --repo-owner $BITBUCKET_WORKSPACE --repo-name $BITBUCKET_REPO_SLUG artifacts: - test-results/** ``` Add `TB_KEY` and `TB_SECRET` as secured repository variables in Bitbucket. Secured variables are encrypted and masked in build logs. ## Frequently asked questions Which CI/CD systems can run Maestro tests on TestingBot? Any CI runner that can execute a shell command. The page below has copy-paste-ready examples for [GitHub Actions](https://testingbot.com#github), [GitLab CI](https://testingbot.com#gitlab), [Jenkins](https://testingbot.com#jenkins), [CircleCI](https://testingbot.com#circleci) and [Bitbucket Pipelines](https://testingbot.com#bitbucket). The same TestingBot CLI works on any other CI runner with a shell. How do I store TestingBot credentials in CI securely? Set TB\_KEY and TB\_SECRET as encrypted CI secrets. GitHub Actions uses repository or environment secrets, GitLab CI uses protected variables, Jenkins uses credentials, CircleCI uses environment variables in contexts, and Bitbucket uses secured repository variables. Never commit the key or secret in plain text. What exit codes does the TestingBot CLI return? Exit code 0 when every flow passed, 2 when one or more flows failed, and 1 for a CLI or infrastructure error such as a failed upload or missing credentials. See the [Exit Codes section](https://testingbot.com#exitcodes) for the full mapping so you can wire conditional follow-up steps in CI. How do I publish Maestro test reports in CI? The CLI writes JUnit, HTML or Allure reports that most CI runners can parse to show per-flow results, and prints a machine-readable JSON document with --json. See the [Test Reports section](https://testingbot.com#reports) for the file path and configuration in each CI system. How do I link CI metadata (branch, commit, pull request) to TestingBot test runs? Pass --repo-owner, --repo-name, --branch, --commit-sha, --pull-request-id and --pr-url flags to the CLI, or use the TestingBot Maestro GitHub Action which fills them in from the workflow context. The values appear in the TestingBot dashboard, and with the TestingBot GitHub App installed they also post a status check on the pull request. See the [CI/CD Metadata section](https://testingbot.com#metadata). Can I check on an async Maestro run later from CI? Yes. Start the run with --async, then use `testingbot status --id --wait` in a later job to block until it finishes, and `testingbot artifacts` to fetch reports and videos. See the [Async runs section](https://testingbot.com#async). --- URL: https://testingbot.com/support/app-automate/maestro/ci-cd/github-actions # Maestro on GitHub Actions The [TestingBot Maestro GitHub Action](https://github.com/marketplace/actions/testingbot-maestro) runs your Maestro flows on TestingBot real devices, emulators and simulators from a workflow. It wraps the [TestingBot CLI](https://testingbot.com/support/app-automate/maestro#cli), fills in branch, commit, repository and pull request from the workflow context, writes a results table to the job summary and exposes every result as an output. Store your credentials in **Settings → Secrets and variables → Actions** as repository secrets. ## Workflow ```yaml name: Maestro Tests on: push: branches: [main] pull_request: branches: [main] jobs: maestro-tests: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Build Android App run: ./gradlew assembleDebug - name: Run Maestro Tests on TestingBot id: maestro uses: testingbot/maestro-action@v1 with: api-key: ${{ secrets.TB_KEY }} api-secret: ${{ secrets.TB_SECRET }} app-file: app/build/outputs/apk/debug/app-debug.apk workspace: .maestro device: Pixel 9 device-version: '14' report: junit download-artifacts: failed - name: Upload Test Results uses: actions/upload-artifact@v4 if: always() with: name: maestro-results path: | testingbot-reports testingbot-artifacts ``` The action fails the step with exit code `2` when a flow fails and `1` on CLI or infrastructure errors. Its outputs let later steps react to the result: | Output | Description | | --- | --- | | `app-id` | Project ID; pass it as `app-binary-id` in later jobs to skip the upload | | `console-url` | Link to the results in the TestingBot dashboard | | `outcome` | `passed`, `failed`, `started` (async), `dry-run` or `error` | | `flow-results` | JSON array of per-flow results with name, status, device, duration and errors | | `results-file` | Path to the full JSON document written by the CLI | Every CLI feature is available as an input: `device-matrix`, `real-device`, `include-tags`, `exclude-flows`, `env`, `retry`, `shard-split`, `async` and more. The inputs mirror the Maestro Cloud action, so migrating is usually a one-line change; see the [action README](https://github.com/testingbot/maestro-action#readme) for the full list. Prefer the CLI directly? The same run without the action: ```yaml - uses: actions/setup-node@v4 with: node-version: '22' - name: Run Maestro Tests run: | npx --yes @testingbot/cli@latest maestro \ app/build/outputs/apk/debug/app-debug.apk ./maestro-flows \ --device "Pixel 9" --deviceVersion "14" \ --report junit --report-output-dir ./test-results \ --commit-sha ${{ github.event.pull_request.head.sha || github.sha }} \ --branch ${{ github.head_ref || github.ref_name }} \ --pull-request-id ${{ github.event.pull_request.number }} \ --repo-owner ${{ github.repository_owner }} \ --repo-name ${{ github.event.repository.name }} env: TB_KEY: ${{ secrets.TB_KEY }} TB_SECRET: ${{ secrets.TB_SECRET }} ``` Add `TB_KEY` and `TB_SECRET` as repository secrets in your GitHub repository settings. Never commit credentials to your repository. ## Triggers and pull requests from forks Run on pushes to your default branch and on pull requests. For `pull_request` events the action uses the pull request's head commit, which is what branch protection evaluates. Pull requests from forks cannot read repository secrets under `pull_request`; if you need them, use `pull_request_target` and check out the PR head explicitly so the tests run against the proposed code rather than the base branch: ```yaml on: push: branches: [main] pull_request_target: branches: [main] jobs: maestro: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 with: ref: ${{ github.event.pull_request.head.sha }} ``` A Linux runner is enough for iOS too: the tests run on TestingBot's simulators and devices, not on the runner, so no macOS minutes are needed unless you build the app in the same job. ## Android, iOS and Expo apps | Platform | `app-file` | | --- | --- | | Android | An `.apk`, for example `app/build/outputs/apk/debug/app-debug.apk`. Emulators by default; add `real-device: true` for a physical device. | | iOS simulator | A simulator `.app` bundle or a `.zip` of it (`xcodebuild -sdk iphonesimulator`). | | iOS real device | An `.ipa`. Real devices are selected automatically for `.ipa` files. | | Expo / EAS | Download the build in a previous step and pass its path, or pass the `eas build` URL with `extra-args: --app-url `. For EAS Workflows use the [EAS integration](https://testingbot.com/support/app-automate/maestro/ci-cd/expo-eas). | ## Inputs | Input | Description | | --- | --- | | `api-key`, `api-secret` | TestingBot credentials. Required. | | `app-file` | App under test (`.apk`, `.ipa`, `.app`, `.zip`, `.tar.gz`). Either this or `app-binary-id`. | | `app-binary-id` | Project ID of an app uploaded earlier (the `app-id` output); skips the upload. | | `workspace` | Flows directory, file or glob; several entries one per line. Default `.maestro`. | | `name` | Run name in the dashboard. Defaults to the PR title, then the commit message. | | `async` | Submit and return immediately; the step never fails on results. | | `timeout` | Minutes to wait for results before cancelling and failing. Default 60. | | `env` | Flow environment variables, one `KEY=VALUE` per line. | | `include-tags`, `exclude-tags`, `exclude-flows` | Flow selection, comma-separated. | | `device`, `device-version`, `platform`, `real-device` | Single device selection. Platform is detected from the app when omitted. | | `device-matrix` | Run every flow on each listed device, `[:][:real]`, one per line. Not with `device`. | | `device-locale`, `timezone`, `orientation`, `google-play` | Device settings. | | `retry`, `shard-split`, `maestro-version` | Retries (0-2), parallel shards per device, Maestro version. | | `report`, `report-output-dir` | `junit`, `html`, `html-detailed` or `allure`, saved to `testingbot-reports` by default. | | `download-artifacts`, `artifacts-output-dir` | `all` or `failed`: logs, screenshots and video as a zip. | | `other-app`, `metadata`, `groups`, `throttle-network`, `geo-country-code`, `tunnel`, `tunnel-identifier` | Companion apps, free-form metadata, dashboard groups, network profile, IP geolocation and the TestingBot Tunnel. | | `check-name` | Names the pull request check `TestingBot / `, for one check per job on the same commit (`iOS`, `Android`). Keep it stable per job. | | `cli-version`, `dry-run`, `extra-args` | Pin the CLI version, validate without running, or append raw CLI arguments. | The [action README](https://github.com/testingbot/maestro-action#readme) lists every input with its default. ## Outputs Give the step an `id` and read its outputs in later steps or jobs: | Output | Description | | --- | --- | | `app-id` | TestingBot project ID. Pass it as `app-binary-id` to skip the upload in a later job. | | `console-url` | Link to the results in the TestingBot dashboard. | | `outcome` | `passed`, `failed`, `started` (async), `dry-run` or `error`. | | `flow-results` | JSON array with one entry per flow attempt: `name`, `status`, `passed`, `attempt`, `latest`, `device`, `durationSeconds`, `errors`, `url`. | | `results-file` | Path to the full JSON document the CLI wrote. | ### Post the result to Slack ```yaml - id: maestro uses: testingbot/maestro-action@v1 with: api-key: ${{ secrets.TB_KEY }} api-secret: ${{ secrets.TB_SECRET }} app-file: app.apk - uses: slackapi/slack-github-action@v2 if: always() with: method: chat.postMessage token: ${{ secrets.SLACK_BOT_TOKEN }} payload: | channel: "C0123456789" text: "Maestro on TestingBot: ${{ steps.maestro.outputs.outcome }} — ${{ steps.maestro.outputs.console-url }}" ``` ### Keep the results as a workflow artifact ```yaml - uses: actions/upload-artifact@v4 if: always() with: name: maestro-results path: | ${{ steps.maestro.outputs.results-file }} testingbot-reports testingbot-artifacts ``` The job summary always gets a results table with a link per flow, so most runs need no extra reporting step. ## Examples ### Upload once, run two suites on different devices ```yaml - id: smoke uses: testingbot/maestro-action@v1 with: api-key: ${{ secrets.TB_KEY }} api-secret: ${{ secrets.TB_SECRET }} app-file: app.apk workspace: .maestro/smoke device: Pixel 9 - uses: testingbot/maestro-action@v1 with: api-key: ${{ secrets.TB_KEY }} api-secret: ${{ secrets.TB_SECRET }} app-binary-id: ${{ steps.smoke.outputs.app-id }} workspace: .maestro/regression device-matrix: | Pixel 9:14 Samsung Galaxy S24:14:real ``` ### Pass secrets to your flows ```yaml - uses: testingbot/maestro-action@v1 with: api-key: ${{ secrets.TB_KEY }} api-secret: ${{ secrets.TB_SECRET }} app-file: app.apk env: | USERNAME=${{ secrets.TEST_USERNAME }} PASSWORD=${{ secrets.TEST_PASSWORD }} include-tags: smoke,critical exclude-tags: wip ``` Flows read them as `${USERNAME}`. GitHub masks secret values in the logs; the action never prints the `env` values itself. ### Fire and forget on pull requests ```yaml - uses: testingbot/maestro-action@v1 with: api-key: ${{ secrets.TB_KEY }} api-secret: ${{ secrets.TB_SECRET }} app-file: app.apk async: true ``` The step returns within seconds and the runner is free while the flows execute on TestingBot. Check on the run later with `testingbot status --id --wait` or in the dashboard. With the TestingBot GitHub App installed the pull request still gets its [status check](https://testingbot.com/support/app-automate/maestro/ci-cd/github-pr-checks) when the run finishes, so merges can be gated without holding a runner. ## Pull request status checks The action passes the repository and the pull request's head commit along, so with the [TestingBot GitHub App](https://testingbot.com/support/app-automate/maestro/ci-cd/github-pr-checks) installed a `TestingBot / tests` check appears on the pull request, listing each flow on each device, and can be made a required check. Running iOS and Android as separate jobs? Set `check-name` on each so they post `TestingBot / iOS` and `TestingBot / Android`. ![TestingBot / tests check on a GitHub pull request for a Maestro run, with the device, flow and result table](https://testingbot.com/assets/support/github-pr-checks/maestro-check-6185df5c10dcff0db4042e811f527a36ba354a07a1fb30f4a9ebb6b2e7c5cd65.png) ## Migrating from the Maestro Cloud action Swap the action and the credentials; the other inputs keep their names. There is no `project-id`. ```yaml - uses: testingbot/maestro-action@v1 with: api-key: ${{ secrets.TB_KEY }} api-secret: ${{ secrets.TB_SECRET }} app-file: app.zip workspace: .maestro include-tags: smoke ``` Outputs map as `MAESTRO_CLOUD_CONSOLE_URL` → `console-url`, `MAESTRO_CLOUD_APP_BINARY_ID` → `app-id`, `MAESTRO_CLOUD_UPLOAD_STATUS` → `outcome` and `MAESTRO_CLOUD_FLOW_RESULTS` → `flow-results`. The full flag mapping is on the [migration page](https://testingbot.com/support/app-automate/maestro/migrate-from-maestro-cloud). ## Frequently asked questions ### How do I run Maestro tests on TestingBot from GitHub Actions? Add the `testingbot/maestro-action@v1` step with your API key and secret as repository secrets, the app file and the flows directory. The action uploads the app and flows, runs them on TestingBot and fails the step when a flow fails. See the [workflow example](https://testingbot.com#workflow). ### Can the action reuse an app uploaded in an earlier step? Yes. Every run exposes an `app-id` output; pass it as `app-binary-id` in a later step or job to skip the upload. See the [examples](https://testingbot.com#examples). ### Does the action post a status check on the pull request? Yes, when the TestingBot GitHub App is installed for the repository. The action passes the head commit and repository along and the check appears automatically. See [GitHub PR Checks for Maestro](https://testingbot.com/support/app-automate/maestro/ci-cd/github-pr-checks). ### Which exit codes does the action step return? The same as the CLI: 2 when one or more flows failed, 1 on a CLI or infrastructure error, 0 otherwise, including `async` and `dry-run`. ### Do I need a macOS runner for iOS? No. The flows run on TestingBot's simulators and real devices, so `ubuntu-latest` is enough for both platforms. Use a macOS runner only if you build the iOS app in the same job. ### How do I run tests on pull requests from forks? Forked pull requests cannot read repository secrets under `pull_request`. Use `pull_request_target` and check out `github.event.pull_request.head.sha` explicitly. See [triggers](https://testingbot.com#triggers). ### Is it a drop-in replacement for the Maestro Cloud action? Mostly. Swap the `uses` line and the credentials; `app-file`, `workspace`, `env`, tags, `async` and the other inputs keep their names. See [migrating](https://testingbot.com#migrate). --- URL: https://testingbot.com/support/app-automate/maestro/ci-cd/github-pr-checks # GitHub PR Checks for Maestro With the TestingBot GitHub App installed, every Maestro run started from a pull request posts a native `TestingBot / tests` status check on it: each flow on each device, failures first, with a link to the run in the TestingBot dashboard. Make it a [required check](https://testingbot.com#merge-gate) and a pull request cannot be merged while a Maestro flow is failing. There is nothing to register or resolve from your workflow; the [Maestro GitHub Action](https://testingbot.com/support/app-automate/maestro/ci-cd/github-actions) and the [TestingBot CLI](https://testingbot.com/support/app-automate/maestro#cli) pass the commit along and TestingBot finalizes the check when every flow has finished. ![TestingBot / tests check on a GitHub pull request for a Maestro run: All 1 flow passed on TestingBot, with a device, flow and result table linking to the run](https://testingbot.com/assets/support/github-pr-checks/maestro-check-6185df5c10dcff0db4042e811f527a36ba354a07a1fb30f4a9ebb6b2e7c5cd65.png) ## Install the GitHub App 1. In the members area, open [Integrations](https://testingbot.com/members/integrations) and choose **GitHub PR Checks**. 2. Click **Install the TestingBot GitHub App**. You are redirected to GitHub. 3. Pick the account or organization and the repositories to enable, then install. You are redirected back and the integration shows as **Connected**. Install through TestingBot rather than from GitHub's app settings, so the installation is linked to your TestingBot account. Only account owners and admins can connect or disconnect the integration. Sub-accounts of a team share the owner's installation: a run started with a sub-account's API key posts checks as long as the app is installed under the team owner. ## Repositories and disconnecting Checks appear only on repositories the app can reach. The integration page shows whether the app covers all repositories or a selection, with a link to change the selection on GitHub; changes apply immediately because TestingBot listens to GitHub's installation events. Disconnecting from the integration page stops checks; reconnecting later reuses the existing installation. ## Run from a pull request With the Maestro GitHub Action a single step is enough; it fills in the repository, head commit, branch and pull request from the workflow context: ```yaml - uses: testingbot/maestro-action@v1 with: api-key: ${{ secrets.TB_KEY }} api-secret: ${{ secrets.TB_SECRET }} app-file: app/build/outputs/apk/debug/app-debug.apk workspace: .maestro ``` From any other CI, pass the metadata flags to the CLI. The commit SHA must be the full 40-character head commit of the pull request, not the merge commit: ```bash npx --yes @testingbot/cli@latest maestro app.apk ./flows \ --repo-owner "$REPO_OWNER" --repo-name "$REPO_NAME" \ --commit-sha "$HEAD_SHA" --branch "$BRANCH" \ --pull-request-id "$PR_NUMBER" ``` Async runs are a good fit: with `--async` (or `async: true` on the action) the CI step returns within seconds, the flows run on TestingBot, and the check is finalized when they finish, so merges are gated without holding a runner. The CLI prints `GitHub check registered for owner/repo@sha` when the link was made. If that line is missing, one of the values was absent, the commit SHA was not the full head SHA, or the app is not installed for the repository owner. In EAS Workflows the [eas-workflow wrapper](https://testingbot.com/support/app-automate/maestro/ci-cd/expo-eas) sets the same flags from `TB_GH_*` variables. ## Pull request comment Alongside the check, TestingBot maintains one self-updating comment on the pull request with the same device, flow and result table. Pushing new commits updates the comment rather than adding new ones. ![TestingBot Maestro results comment on a pull request: all flows passed, with the device, flow and result table](https://testingbot.com/assets/support/github-pr-checks/maestro-pr-comment-b4a0e71c87de95bf678fbc8976eca5f497697a2778f0de0d0a156b0556ac65c4.png) ## Require green flows before merging 1. In your GitHub repository, go to **Settings** → **Branches** (or **Rules** → **Rulesets** ). 2. Add a rule for your default branch and enable **Require status checks to pass before merging**. 3. Select `TestingBot / tests` as a required check. GitHub lists it once it has run on the branch at least once in the last 7 days, so open one pull request first. The check is green only when every run finished and every flow passed. A cancelled run, a failed flow or a run that never completed all conclude as failing, so the merge stays blocked until the tests are genuinely green. Flows that failed and then passed on a `--retry` count as passed; only the newest attempt per flow is judged, the same rule the dashboard and the CLI exit code use. With `--device-matrix` every device is a row in the same check. ## Several runs on one pull request Each Maestro run posts one check, named `TestingBot / tests` by default. When more than one run tests the same commit, an iOS job and an Android job for example, give each a check name so they post separate checks that branch protection can require individually: `--check-name iOS` on the CLI, `check-name: iOS` on the action, or `TB_CHECK_NAME=iOS` with the EAS wrapper. The check is then called `TestingBot / iOS` and gets its own pull request comment. ```yaml jobs: ios: steps: - uses: testingbot/maestro-action@v1 with: app-file: app.zip check-name: iOS android: steps: - uses: testingbot/maestro-action@v1 with: app-file: app.apk check-name: Android ``` Keep the name stable per job: GitHub matches required checks by name, so a name derived from the commit or run number would create a new check every time and could never be required. Two runs on the same commit with the same name share one check, and the last to finish decides its result. To cover several devices in one job, prefer `--device-matrix`; every device becomes a row in the same check. ## Troubleshooting - **No check appears.** Look for `GitHub check registered` in the CLI output. Without it, check that the repository owner and name and a full 40-character commit SHA were passed (the Maestro GitHub Action does this for you) and that the app is installed for the repository owner. - **The check is on the wrong commit.** For `pull_request` events pass `github.event.pull_request.head.sha`; `github.sha` is the merge commit, which is not part of the pull request. - **The check is stuck as in-progress.** It finalizes when every run and flow has settled. If a run was cancelled or lost, a watchdog concludes the check as failing after about 50 minutes so the pull request is never left waiting. - **The check is not selectable as a required check.** GitHub only offers it after it has run at least once on the protected branch in the last 7 days. - **The run was started with another team's key.** The check posts only when the API key belongs to the team that installed the app. Check which key your CI secrets hold. - **Re-running after a failure.** The check carries a **Re-run analysis** button that re-evaluates the current results. To re-run the flows themselves, use `--retry` on the run or re-run the CI job; each new run updates the check. Browser and Appium builds use the same app with a different registration flow; see [GitHub PR Checks](https://testingbot.com/support/integrations/ci-cd/github-pr-checks). ## Frequently asked questions ### Do I need to register the build from my workflow? No. Unlike browser builds, a Maestro run registers itself: when it carries the repository and the pull request head commit, and the app is installed, the check is created when the run starts and finalized when every flow has finished. ### When is the check green? Only when every run finished and every flow passed. Cancelled, failed or incomplete runs conclude as failing so the merge stays blocked. A flow that passed on a retry counts as passed. See [the merge gate](https://testingbot.com#merge-gate). ### Does it work with --async? Yes. The check is created when the run starts and finalized by TestingBot when the run completes, whether or not the CLI is still waiting. ### Can I use it from CI other than GitHub Actions? Yes. Pass `--repo-owner`, `--repo-name`, the full `--commit-sha` and optionally `--branch` and `--pull-request-id` to the CLI from any CI. See [run from a pull request](https://testingbot.com#run). ### Can I have one check per platform on the same pull request? Yes. Give each run a check name (`--check-name iOS`, the action's `check-name` input or `TB_CHECK_NAME` in EAS) and it posts as `TestingBot / iOS` with its own comment, so branch protection can require each platform separately. See [several runs on one pull request](https://testingbot.com#one-check). ### Why does my Maestro check not appear? The CLI prints `GitHub check registered` when the link was made. Without that line, the metadata was incomplete, the commit SHA was not the full head SHA, or the app is not installed for the repository owner. See [troubleshooting](https://testingbot.com#troubleshooting). --- URL: https://testingbot.com/support/app-automate/maestro/ci-cd/expo-eas # Maestro tests for Expo apps on TestingBot With Expo there is no local binary: `eas build` produces a download URL, a `.tar.gz` for iOS simulator builds and an `.apk` for Android. The TestingBot CLI takes that URL directly, and inside EAS Workflows a wrapper turns the results into step outputs. Both paths run your flows on TestingBot real devices, emulators and simulators with no Maestro Cloud account. ## Prerequisites and secrets - An Expo project with an EAS build profile for the platform you test. iOS flows need a **simulator** build (`"ios": { "simulator": true }` in `eas.json`) unless you run on real devices with an `.ipa`. - Maestro flows in the repository, by convention under `.maestro/`. - Your TestingBot key and secret from the [dashboard](https://testingbot.com/members/user/edit). In EAS Workflows a `linux-medium` runner is enough for both platforms, because the flows run on TestingBot's simulators and devices; no macOS runner is needed. Store the credentials as EAS project secrets once. EAS injects them into every step environment; the `${{ secrets.X }}` syntax does not exist in EAS Workflows. ```bash eas env:create --scope project --name TB_KEY --value --visibility secret --type string \ --environment production --environment preview --environment development eas env:create --scope project --name TB_SECRET --value --visibility secret --type string \ --environment production --environment preview --environment development ``` Secrets your flows need, such as a test user's password, are created the same way and passed with `--env "PASSWORD=$TEST_PASSWORD"`. ## From any CI: eas build and --app-url With Expo there is no local binary: `eas build` produces a download URL, a `.tar.gz` for iOS simulator builds and an `.apk` for Android. From any CI, pass that URL with `--app-url`; the CLI downloads it and, for iOS, extracts the `.app` before uploading: ```bash URL=$(eas build --platform ios --profile preview --json --non-interactive | jq -r '.[0].artifacts.buildUrl') npx --yes @testingbot/cli@latest maestro --app-url "$URL" ./flows --device "iPhone 16" ``` ## Inside EAS Workflows Inside [EAS Workflows](https://docs.expo.dev/eas/workflows/get-started/) the build is already on the runner after `eas/download_build`, so pass the artifact path. Credentials come from EAS project secrets (`eas env:create --name TB_KEY --visibility secret …`), which EAS injects into the step environment; the `${{ secrets.X }}` syntax is not available there. The [@testingbot/eas-workflow](https://github.com/testingbot/testingbot-eas) wrapper turns the results into step outputs for `github-comment` and `slack` jobs: [Android](https://testingbot.com#) [iOS](https://testingbot.com#) ```yaml # .eas/workflows/maestro-android.yml name: Maestro E2E (Android) on: pull_request: {} jobs: build_android: type: build params: platform: android profile: preview e2e_android: needs: [build_android] runs_on: linux-medium # Job-level env: EAS may silently ignore env set on a step. env: TB_EAS_BUILD_ID: ${{ needs.build_android.outputs.build_id }} TB_EAS_PLATFORM: android TB_EAS_PROFILE: preview TB_GH_SHA: ${{ github.sha }} TB_GH_BRANCH: ${{ github.ref_name }} TB_GH_REPO_NAME: ${{ github.repository }} outputs: console_url: ${{ steps.testingbot.outputs.console_url }} run_status: ${{ steps.testingbot.outputs.run_status }} summary: ${{ steps.testingbot.outputs.summary }} steps: - uses: eas/checkout - uses: eas/download_build id: download with: build_id: ${{ needs.build_android.outputs.build_id }} - id: testingbot run: | npx --yes @testingbot/eas-workflow@v1 \ --app-file ${{ steps.download.outputs.artifact_path }} \ --flows ./.maestro \ --device "Pixel 9" --deviceVersion "14" \ --report junit --report-output-dir ./reports report: after: [e2e_android] type: github-comment params: payload: | ### Maestro on TestingBot — ${{ after.e2e_android.outputs.run_status }} ${{ after.e2e_android.outputs.summary }} [View results](${{ after.e2e_android.outputs.console_url }}) ``` ```yaml # .eas/workflows/maestro-ios.yml name: Maestro E2E (iOS) on: pull_request: {} jobs: build_ios: type: build params: platform: ios profile: preview # a simulator profile: "ios": { "simulator": true } e2e_ios: needs: [build_ios] runs_on: linux-medium env: TB_EAS_BUILD_ID: ${{ needs.build_ios.outputs.build_id }} TB_EAS_PLATFORM: ios TB_EAS_PROFILE: preview TB_GH_SHA: ${{ github.sha }} TB_GH_BRANCH: ${{ github.ref_name }} TB_GH_REPO_NAME: ${{ github.repository }} outputs: console_url: ${{ steps.testingbot.outputs.console_url }} run_status: ${{ steps.testingbot.outputs.run_status }} summary: ${{ steps.testingbot.outputs.summary }} steps: - uses: eas/checkout - uses: eas/download_build id: download with: build_id: ${{ needs.build_ios.outputs.build_id }} - id: testingbot run: | npx --yes @testingbot/eas-workflow@v1 \ --app-file ${{ steps.download.outputs.artifact_path }} \ --flows ./.maestro \ --device "iPhone 16" --deviceVersion "18" report: after: [e2e_ios] type: github-comment params: payload: | ### Maestro on TestingBot — ${{ after.e2e_ios.outputs.run_status }} ${{ after.e2e_ios.outputs.summary }} [View results](${{ after.e2e_ios.outputs.console_url }}) ``` `after` rather than `needs` on the report job means the comment is still posted when the e2e job fails, which is exactly when you want it. Downstream jobs read the outputs as `needs.e2e_android.outputs.console_url`; the same values feed a `slack` job. The CLI records the EAS build id, profile, platform and commit on the run automatically when it runs inside an EAS job, and the `TB_GH_*` variables add the repository and pull request so a [GitHub status check](https://testingbot.com/support/integrations/ci-cd/github-pr-checks#maestro) is posted too. EAS download links are signed and expire after about an hour; a rejected link is reported as such so you know to request a fresh one. A local `.tar.gz` can also be passed as the app file directly, and `testingbot upload ` uploads once for reuse with `--app-binary-id`. ## Pitfalls specific to EAS Workflows - **Set `env` on the job, not the step.** Step-level `env` can be ignored silently by EAS. - **Pull request expressions are `null` outside pull requests.** `github.event.pull_request.number` evaluates to `null` on a manual or push trigger and EAS rejects the job. Set `TB_GH_PR_NUMBER` and `TB_GH_PR_URL` only in a workflow that runs on `pull_request`, or leave them out; `github.sha` and `github.ref_name` safely become empty strings. - **`github.repository` is `owner/repo`.** Pass it as `TB_GH_REPO_NAME`; the wrapper splits it, so `TB_GH_REPO_OWNER` is optional. - **Never name a variable `EAS_BUILD_*`.** That namespace belongs to the EAS Build worker; overwriting `EAS_BUILD_ID` ends the job silently after the prepare phase. The wrapper's variables use the `TB_` prefix for this reason. - **Build links expire.** An `eas build` URL is signed and stops working after about an hour; inside EAS Workflows use `eas/download_build` instead of a stored URL. ## Wrapper options and outputs `@testingbot/eas-workflow` takes `--app-file`, `--app-url` or `--app-binary-id` plus `--flows`; every other flag is forwarded to `testingbot maestro`, so `--device`, `--device-matrix`, `--retry`, `--report` and the rest all work. Step outputs: `console_url`, `app_id`, `run_urls`, `run_status`, `outcome`, `total_flows_count`, `successful_flows_count`, `failed_flows_count`, the flow name lists and a one-line `summary`. The step exits 2 when a flow fails and 1 on a CLI or infrastructure error. Environment variables: `TB_KEY` and `TB_SECRET` (required), `TB_GH_SHA`, `TB_GH_BRANCH`, `TB_GH_PR_NUMBER`, `TB_GH_PR_URL`, `TB_GH_REPO_OWNER` and `TB_GH_REPO_NAME` for the run's CI metadata and [PR status check](https://testingbot.com/support/app-automate/maestro/ci-cd/github-pr-checks), `TB_CHECK_NAME` to name that check per job (`iOS`, `Android`), `TB_RUN_NAME` and `TB_GROUPS` for the dashboard, `TB_CLI_VERSION` to pin the CLI. Never name a variable `EAS_BUILD_*`; that namespace belongs to the EAS Build worker. Full reference and complete Android and iOS workflow files: [testingbot/testingbot-eas](https://github.com/testingbot/testingbot-eas). ## Migrating from the maestro-cloud job type Replace the pre-packaged job with a custom job. No Maestro Cloud project id or API key is needed. ```yaml # Before jobs: e2e: needs: [build_android] type: maestro-cloud params: build_id: ${{ needs.build_android.outputs.build_id }} maestro_project_id: proj_abc123 flows: ./.maestro # After jobs: e2e: needs: [build_android] steps: - uses: eas/checkout - uses: eas/download_build id: download with: build_id: ${{ needs.build_android.outputs.build_id }} - id: testingbot run: | npx --yes @testingbot/eas-workflow@v1 \ --app-file ${{ steps.download.outputs.artifact_path }} \ --flows ./.maestro ``` `include_tags`, `exclude_tags`, `maestro_version`, `maestro_config`, `device_locale`, `name` and `async` become the CLI flags of the same meaning; `device_model` and `device_os` become `--device` and `--deviceVersion`. Output names carry over, only `maestro_cloud_url` is renamed to `console_url`, so downstream `github-comment` and `slack` jobs need no changes. ## Frequently asked questions ### How do I run Maestro flows against an EAS Build? Pass the build URL with `--app-url`. The CLI downloads it and, for an iOS simulator build, extracts the `.app` from the `.tar.gz` before uploading. See [eas build and --app-url](https://testingbot.com#app-url). ### How do I use it inside EAS Workflows? Download the build with `eas/download_build` and run `npx --yes @testingbot/eas-workflow@v1 --app-file ... --flows ...`. Credentials come from EAS project secrets. See [inside EAS Workflows](https://testingbot.com#workflows). ### Why can I not use ${{ secrets.TB\_KEY }} in an EAS workflow? EAS Workflows do not support that syntax. Create `TB_KEY` and `TB_SECRET` with `eas env:create --visibility secret`; EAS injects them into the step environment. ### Do I need a macOS runner for iOS flows? No. The flows run on TestingBot's simulators and devices; `linux-medium` is enough for both platforms. The iOS build job itself still runs on EAS's macOS workers. ### Why does my job fail with a null expression? `github.event.pull_request.*` is `null` on manual and push triggers and EAS rejects it. Keep those variables in workflows triggered by `pull_request` only. See [pitfalls](https://testingbot.com#pitfalls). ### Does an EAS Build link expire? Yes, after about an hour. The CLI reports a rejected link as expired so you know to request a fresh one, for example by re-running `eas build --json`. ### Can I replace the maestro-cloud job type? Yes. Swap it for a custom job with `eas/download_build` and the wrapper; parameters map to CLI flags of the same meaning and the outputs keep their names. See [migrating](https://testingbot.com#migrate). --- URL: https://testingbot.com/support/app-automate/maestro/tunnel # Run Maestro with the TestingBot Tunnel TestingBot runs your Maestro flows on real devices, simulators and emulators in our cloud datacenter. Those devices cannot, by default, reach an app or API that lives on your machine, your CI runner or a private staging network. The [TestingBot Tunnel](https://testingbot.com/support/tunnel) bridges that gap: it opens a secure, SSH-encrypted connection so the device under test can reach your internal hosts as if it were sitting on your own network. The easiest way to use a tunnel with Maestro is the `-t` flag on the [TestingBot CLI](https://testingbot.com/support/app-automate/maestro#cli). It starts a tunnel, runs your flows through it, and tears the tunnel down when the run completes. ## When you need a tunnel Use a tunnel whenever your Maestro flow drives an app that talks to a backend the public internet cannot see. Common scenarios: - **Staging and pre-production backends.** Your app points at a staging API (`https://staging.internal.example.com`) that is firewalled off from the public internet. The tunnel lets the cloud device resolve and reach it. - **Localhost development servers.** You are iterating locally and your API runs on `http://localhost:3000` on your laptop or CI runner. The tunnel forwards that port to the device. - **Protected or token-gated endpoints.** Your backend requires a session or bearer token that is only issued inside your network. A small [helper script](https://testingbot.com#token) can `fetch` that token from a protected endpoint over the tunnel at runtime, so the secret never lives in your flow files. - **Private databases and internal services.** The app calls an internal microservice, payment sandbox or database proxy that has no public DNS entry. - **Mock and contract-test servers.** You run a mock server (WireMock, Mockoon, MSW) on your CI runner to return deterministic responses; the tunnel exposes it to the device for stable, hermetic tests. - **Geo or VPN-restricted APIs.** An upstream API only accepts traffic from your office or VPN egress IP. Routing through the tunnel makes requests originate from your network. If your backend is already publicly reachable (a deployed staging URL with valid TLS and no IP allowlist), you do not need a tunnel. The tunnel is only required when the device cannot otherwise reach the host. ## Quick Start Add the `-t` flag (short for `--tunnel`) to your normal Maestro run command. The CLI starts a tunnel before the run and stops it afterwards: ```bash testingbot maestro app.apk ./flows \ --device "Pixel 8" \ --deviceVersion "14" \ -t ``` That single command will: - Authenticate and start a TestingBot Tunnel for this run - Upload your app and Maestro flows - Run the flows, routing all device traffic through the tunnel - Shut the tunnel down once the run finishes | Flag | Description | | --- | --- | | `-t, --tunnel` | Start a TestingBot Tunnel for this test run | | `--tunnel-identifier ` | Identifier for the tunnel, allowing multiple tunnels to run in parallel | The `-t` flag cannot be combined with `--async`. The tunnel has to stay open for the whole run, so the CLI must wait for the flows to finish. Prefer to manage the tunnel yourself (for example, to keep one tunnel up across many runs)? Start the [TestingBot Tunnel](https://testingbot.com/support/tunnel) client separately and leave it running. When a tunnel is active on your account, Maestro runs automatically route through it, with no extra flags needed. ## How it works When the tunnel is up, the cloud device sends its network requests over the encrypted tunnel to the tunnel client running in your environment. The client forwards each request to the target host (your localhost, staging server or internal API) and relays the response back to the device. ![TestingBot Tunnel architecture diagram](https://testingbot.com/assets/features/tunnel-diagram-poster-8c25e19e6fe4f7276a46a96b31a0ad0e07bb4f5bf98dc192d2b70897cfbb1a26.png) Because the request originates from inside your network, internal DNS, VPN routes and IP allowlists all behave as they normally would for local traffic. The tunnel supports HTTP/1, HTTP/2, WebSocket and Server-Sent Events, so streaming and real-time backends work as well. For WebSocket or Server-Sent Events backends, start the standalone tunnel client with the `--nobump --nocache` flags. See the [tunnel command reference](https://testingbot.com/support/tunnel/commandline). ## Running multiple tunnels If you run several Maestro builds in parallel, give each one its own named tunnel so traffic routes to the correct environment. Use `--tunnel-identifier` to set a unique name: ```bash # Build A, pointing at the "staging" tunnel testingbot maestro app.apk ./flows/smoke \ --device "Pixel 8" \ -t --tunnel-identifier staging # Build B, pointing at the "qa" tunnel testingbot maestro app.apk ./flows/regression \ --device "iPhone 16" --real-device \ -t --tunnel-identifier qa ``` Without an identifier, all traffic flows through the single default tunnel on your account. See [running multiple tunnels](https://testingbot.com/support/tunnel/multiple) for more detail. ## Staging token helper A very common need: your staging or pre-production backend is protected and the app needs a short-lived auth token before it can do anything useful. You do not want to hardcode that token (it rotates, and committing secrets to flow files is unsafe). Instead, fetch it at runtime from a token endpoint that is only reachable over the tunnel. Maestro can run inline JavaScript with the `runScript` command. The script executes in Maestro's JS environment and can make an HTTP request to your protected backend, which is reachable because the device routes through the tunnel. Store the result in `output` so later steps can use it. ### 1. The helper script Create a small script, for example `scripts/fetch_token.js`, that calls your protected token endpoint: ```javascript // scripts/fetch_token.js // Runs inside Maestro. The request reaches your staging backend // through the TestingBot Tunnel, so localhost / internal hosts resolve. var response = http.post('http://localhost:3000/internal/auth/token', { headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ client: 'maestro-e2e', scope: 'staging' }) }) var token = json(response.body).access_token // Expose the token to the rest of the flow output.authToken = token ``` Maestro exposes an `http` helper and a `json()` parser inside `runScript`. If your backend is on your laptop, the host is `localhost`; for a named staging host, use its internal hostname (for example `http://staging.internal:8080`) and it will resolve over the tunnel. ### 2. Call it from your flow Run the helper near the start of the flow, then use the token in subsequent steps, for example to deep-link into an authenticated screen or to set it in app state: ```yaml appId: com.example.app --- - runScript: scripts/fetch_token.js # Use the token captured by the helper. # For example, open a deep link that carries the session token: - openLink: "exampleapp://session?token=${output.authToken}" - assertVisible: "Welcome back" ``` ### 3. Run it through the tunnel Run as usual with the `-t` flag so `localhost` (or your staging host) is reachable from the device: ```bash testingbot maestro app.apk ./flows \ --device "Pixel 8" \ --deviceVersion "14" \ -t ``` Keep the token endpoint scoped to non-production data and issue short-lived tokens. The tunnel is encrypted end to end, but treat any credential the flow can read as test-grade only. The same pattern works for any runtime value the device needs from a protected source: feature-flag overrides, a seeded test-user id, a one-time passcode from a mailbox API, or a CSRF token. ## Using the tunnel in CI/CD The `-t` flag behaves identically in CI. Build your app, then run Maestro with the tunnel so the cloud device can reach the mock server or staging backend running on the CI runner: ```bash npx --yes @testingbot/cli@latest maestro \ app/build/outputs/apk/debug/app-debug.apk \ ./maestro-flows \ --device "Pixel 8" \ --deviceVersion "14" \ -t \ --report junit \ --report-output-dir ./test-results ``` On GitHub Actions, the [TestingBot Maestro action](https://github.com/testingbot/maestro-action) starts the tunnel for you with `tunnel: true`: ```yaml - uses: testingbot/maestro-action@v1 with: api-key: ${{ secrets.TB_KEY }} api-secret: ${{ secrets.TB_SECRET }} app-file: app/build/outputs/apk/debug/app-debug.apk workspace: ./maestro-flows device: Pixel 8 device-version: '14' tunnel: true report: junit ``` For full GitHub Actions, GitLab CI, Jenkins, CircleCI and Bitbucket Pipelines examples, including secret handling and reports, see the [Maestro CI/CD integration page](https://testingbot.com/support/app-automate/maestro/ci-cd). Because `-t` cannot be combined with `--async`, the CI step blocks until the run finishes, which is what you want so the pipeline can pass or fail on the result. ## CLI Reference | CLI Option | Description | | --- | --- | | `-t, --tunnel` | Start a [TestingBot Tunnel](https://testingbot.com/support/tunnel) for this test run (cannot be combined with `--async`) | | `--tunnel-identifier ` | Identifier for the tunnel, allowing multiple tunnels to run in parallel | See the full [Maestro options](https://testingbot.com/support/app-automate/maestro/options#tunnel) and the [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for advanced settings such as upstream proxies, custom DNS, header injection and PAC support. ## Frequently asked questions Why do I need a tunnel to run Maestro on TestingBot? TestingBot devices run in our cloud datacenter and cannot reach apps or APIs that live on your laptop, your CI runner or a private staging network. The [TestingBot Tunnel](https://testingbot.com/support/tunnel) opens a secure, SSH-encrypted connection so the device under test can resolve and reach those internal hosts as if it were on your own network. See the [use cases section](https://testingbot.com#usecases) for common scenarios. How do I start a tunnel for a Maestro run? Pass the `-t` (or `--tunnel`) flag to the [TestingBot CLI](https://testingbot.com/support/app-automate/maestro#cli). The CLI starts a tunnel, runs your flows through it, and shuts the tunnel down when the run finishes. See the [quick start](https://testingbot.com#quickstart). Can I run multiple Maestro tunnels in parallel? Yes. Give each tunnel a unique name with `--tunnel-identifier ` so parallel runs route through the correct tunnel. See [running multiple tunnels](https://testingbot.com#multiple). How does my app get a staging auth token during a Maestro flow? Your app, running on the cloud device, makes its normal network calls through the tunnel back to your protected backend. A small helper that performs a `fetch`/AJAX call to a token endpoint behind the tunnel can retrieve a short-lived token at runtime, so you never hardcode secrets in the flow. See the [token helper example](https://testingbot.com#token). Can I use the tunnel in CI/CD? Yes. The `-t` flag works the same way in any CI runner. Combine it with the examples on the [Maestro CI/CD page](https://testingbot.com/support/app-automate/maestro/ci-cd). The `-t` flag cannot be combined with `--async`, because the tunnel must stay up for the duration of the run. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/maestro/migrate-from-maestro-cloud # Migrate from Maestro Cloud to TestingBot Maestro flows are plain YAML and are not tied to any one runner. If you already run them on [Maestro Cloud](https://maestro.dev/cloud), moving them to TestingBot is a change of upload command and device flags, not a rewrite. Your flow files, your tags and your Maestro syntax stay as they are. This page maps every `maestro cloud` option to its TestingBot equivalent, so you can port a working pipeline in one sitting. New to Maestro on TestingBot? Start at [Maestro testing](https://testingbot.com/support/app-automate/maestro) for the full setup, then come back here for the mapping. ## Before you start Install the TestingBot CLI and authenticate once: ```bash npm install -g @testingbot/cli testingbot login ``` Credentials are stored in `~/.testingbot`. In CI, pass `--api-key` and `--api-secret`, or set `TB_KEY` and `TB_SECRET` as secrets. ## Command mapping The whole migration in one line. On Maestro Cloud: ```bash maestro cloud --app-file app.apk --flows ./flows ``` On TestingBot: ```bash testingbot maestro app.apk ./flows --device "Pixel 8" --deviceVersion "14" ``` The app binary and the flows are positional arguments rather than named flags, but the `maestro cloud` spelling is accepted too: `--app-file`, `--flows`, `--apiKey`, `--device-model`, `--device-os`, `--format`, `--output` and `--test-suite-name` work as hidden aliases, so an existing command line runs unchanged once you add `--api-secret`: ```bash testingbot maestro --app-file app.apk --flows ./flows --device-model Pixel-7 --device-os android-34 --format JUNIT --output report.xml ``` Everything the flows directory can contain stays the same, and you can pass a directory, several directories, individual flow files, or a pre-zipped bundle: ```bash # A directory of flows testingbot maestro app.apk ./flows # Several directories testingbot maestro app.apk ./flows/smoke ./flows/regression # Individual flow files testingbot maestro app.apk login.yaml checkout.yaml # A pre-zipped bundle testingbot maestro app.apk flows.zip ``` ## Flag mapping Every `maestro cloud` option and what to use instead. Flag names that are identical on both sides are marked, so you can leave those parts of your command untouched. | Maestro Cloud | TestingBot CLI | Notes | | --- | --- | --- | | `--app-file` | first positional argument | Same binary formats: .apk, .aab, .ipa, .app. `--app-file` also works as an alias | | `--flows` | second positional argument | Directory, files or zip. `--flows a,b` also works as an alias | | `--device-model` | `--device` | For example `--device "Pixel 8"`. `--device-model iPhone-17-Pro` is accepted and mapped | | `--device-os` | `--deviceVersion` and `--platform` | Split into two flags, or keep `--device-os iOS-18-2` / `android-34`: it is mapped (Android API levels become OS versions) | | `--device-locale` | `--device-locale` | Identical | | `--async` | `--async` | Identical | | `--include-tags` | `--include-tags` | Identical | | `--exclude-tags` | `--exclude-tags` | Identical | | `-e`, `--env` | `--env` | Same `KEY=value` form | | `--name` | `--name` | Identical | | `--format`, `--output` | `--report`, `--report-output-dir` | See [reports](https://testingbot.com#reports). Both are accepted as aliases; `--report allure` is available too | | `--api-key` | `--api-key` and `--api-secret` | TestingBot uses a key and secret pair | | `--branch` | `--branch` | Identical; `--pr-url` and `-m KEY=VALUE` metadata are available too | | `--commit-sha` | `--commit-sha` | Identical | | `--pull-request-id` | `--pull-request-id` | Identical | | `--repo-name` | `--repo-name` | Identical | | `--repo-owner` | `--repo-owner` | Identical | | `--app-binary-id` | `--app-binary-id` | Identical. `testingbot upload app.apk` prints the ID; see [Reuse an Uploaded App](https://testingbot.com/support/app-automate/maestro/options#app-binary-id) | | `--test-suite-name` | `--name` | Accepted as an alias | | `--android-api-level`, `--ios-version` | `--deviceVersion` | Deprecated upstream in favour of `--device-os` | TestingBot adds flags that have no `maestro cloud` counterpart, including `--device-matrix`, `--json`, `--exclude-flows`, `--retry`, `--shard-split`, `--real-device`, `--tunnel`, `--orientation`, `--timezone`, `--geo-country-code`, `--throttle-network`, `--download-artifacts` and `--maestro-version`. They are documented in [Maestro options](https://testingbot.com/support/app-automate/maestro/options). ## Device and OS selection Maestro Cloud selects an environment with a single combined string, for example `--device-os "android-34"`, or an OS plus model pair on iOS. TestingBot splits that into a device name and a version: ```bash # Android testingbot maestro app.apk ./flows --device "Pixel 8" --deviceVersion "14" # iOS simulator testingbot maestro app.ipa ./flows --device "iPhone 16" --deviceVersion "18.6" --platform "iOS" # Physical iPhone testingbot maestro app.ipa ./flows --device "iPhone 16" --real-device ``` Add `--real-device` to run on physical hardware instead of a simulator or emulator. The full list of device names and versions is on [Maestro options](https://testingbot.com/support/app-automate/maestro/options). ## Tags and environment variables These copy across unchanged, both the flag names and their behaviour: ```bash testingbot maestro app.apk ./flows \ --device "Pixel 8" \ --include-tags smoke,checkout \ --exclude-tags flaky \ --env USERNAME=demo --env BASE_URL=https://staging.example.com ``` Tags still come from the `tags:` block in your flow files, so no flow edits are needed. ## Reports and artifacts Maestro Cloud writes a report with `--format` and `--output`. TestingBot uses a report type and an output directory: ```bash testingbot maestro app.apk ./flows --device "Pixel 8" \ --report junit \ --report-output-dir ./reports ``` Video, logs and screenshots can be pulled down in the same run: ```bash testingbot maestro app.apk ./flows --device "Pixel 8" \ --download-artifacts \ --artifacts-output-dir ./artifacts ``` ## Running flows in parallel Split a flows directory across concurrent sessions with `--shard-split`. Your plan's concurrency decides how many run at once: ```bash # Three parallel sessions testingbot maestro app.apk ./flows \ --device "Pixel 8" --deviceVersion "14" --shard-split 3 # Everything in one session testingbot maestro app.apk ./flows \ --device "Pixel 8" --deviceVersion "14" --shard-split 1 ``` ## CI/CD On GitHub Actions, swap the action and the credentials; the other inputs keep their names: ```yaml - uses: testingbot/maestro-action@v1 with: api-key: ${{ secrets.TESTINGBOT_KEY }} api-secret: ${{ secrets.TESTINGBOT_SECRET }} app-file: app.zip workspace: .maestro include-tags: smoke ``` There is no `project-id`. The outputs map as `MAESTRO_CLOUD_CONSOLE_URL` → `console-url`, `MAESTRO_CLOUD_APP_BINARY_ID` → `app-id`, `MAESTRO_CLOUD_UPLOAD_STATUS` → `outcome` and `MAESTRO_CLOUD_FLOW_RESULTS` → `flow-results`. Maestro Cloud's native pull request integration is covered by [GitHub PR status checks](https://testingbot.com/support/app-automate/maestro/ci-cd/github-pr-checks). From any other CI, the metadata flags keep the same names, so the parts of your pipeline that tag a run with a branch, commit or pull request need no edits: ```bash npx --yes @testingbot/cli@latest maestro app.apk ./flows \ --device "Pixel 8" \ --deviceVersion "14" \ --api-key $TB_KEY \ --api-secret $TB_SECRET \ --repo-owner "$GITHUB_REPOSITORY_OWNER" \ --commit-sha "$GITHUB_SHA" ``` Ready-made GitHub Actions, GitLab CI, Jenkins, CircleCI and Bitbucket Pipelines examples are on [Maestro CI/CD integration](https://testingbot.com/support/app-automate/maestro/ci-cd). If your app talks to a staging environment that is not reachable from the internet, see [local and staging testing](https://testingbot.com/support/app-automate/maestro/tunnel). ## What differs Worth knowing before you port a pipeline: - **Two positional arguments.** The app and the flows are passed by position rather than through `--app-file` and `--flows`. - **Device and version are separate flags.** There is no single combined `--device-os` string. - **Authentication is a key and secret pair** , not a single API key. - **Real hardware is opt-in** with `--real-device`. - **Reports are configured by type and directory** rather than format and file path. ## Frequently asked questions ### Do I have to rewrite my Maestro flows to run them on TestingBot? No. Flows are plain Maestro YAML and run unchanged. What changes is the command that uploads and runs them, and the flags that pick a device. Your flow files, tags and Maestro syntax stay exactly as they are. ### What replaces the maestro cloud command on TestingBot? The TestingBot CLI: `testingbot maestro `. See the [command mapping](https://testingbot.com#commands) for a side-by-side, and the [flag mapping](https://testingbot.com#flags) for every option. ### How do I pick a device and OS version on TestingBot? With `--device` and `--deviceVersion`, instead of a single combined `--device-os` string. Add `--real-device` to run on physical hardware rather than a simulator or emulator. See [device and OS selection](https://testingbot.com#devices). ### Can I keep my --include-tags and --exclude-tags filters? Yes, both flags exist on the TestingBot CLI under the same names and behave the same way, so tag filters copy across unchanged. See [tags and environment variables](https://testingbot.com#tags). ### Will my CI pipeline still publish JUnit reports? Yes. Where Maestro Cloud uses `--format` and `--output`, TestingBot uses `--report junit` and `--report-output-dir`. The CI metadata flags for branch, commit and pull request keep the same names. See [reports and artifacts](https://testingbot.com#reports). Run your existing Maestro flows on TestingBot with a [free trial](https://testingbot.com/users/sign_up), or read why teams choose us on the [Maestro Cloud alternative](https://testingbot.com/maestro-cloud-alternative) page. --- URL: https://testingbot.com/support/app-automate/mobilewright # Mobilewright Testing [Mobilewright](https://github.com/mobile-next/mobilewright) is an open source mobile UI test framework with a Playwright-style API. You write tests in TypeScript or JavaScript against `device`, `screen` and `expect` fixtures, and Mobilewright drives your iOS or Android app. TestingBot maintains [`@testingbot/mobilewright-driver`](https://github.com/testingbot/mobilewright-testingbot), which runs those same tests on our device cloud instead of a device attached to your laptop: real iPhones, iPads and Android phones and tablets in our European datacenter, plus iOS simulators and Android emulators. - No local Appium server, Xcode or Android SDK required. - Your app is uploaded and pre-installed on the device before the first test runs. - Every device slot becomes a TestingBot session with video, command logs and device logs. - Pass and fail verdicts, build names and git metadata are reported for you. The driver requires **Node.js 18 or newer** and **Mobilewright 0.0.53 or newer** , the first release that accepts driver instances. Its peer dependency is `@mobilewright/protocol >=0.0.53 <0.1.0`, which Mobilewright installs for you. ## Quickstart The fastest way to start is the scaffolder. It asks a few questions and writes a `mobilewright.config.ts` plus a sample test into your project. Add `--yes` to accept the defaults. ```bash npx @testingbot/mobilewright-driver init ``` To set things up by hand instead, install the packages as dev dependencies: ```bash npm install --save-dev mobilewright @mobilewright/test @testingbot/mobilewright-driver ``` ### Credentials The driver reads your TestingBot key and secret from the environment. You can find them on your [member edit page](https://testingbot.com/members/user/edit). ```bash export TESTINGBOT_KEY=your_key export TESTINGBOT_SECRET=your_secret ``` `TB_KEY` and `TB_SECRET` are accepted as aliases. If you would rather not use environment variables, pass `key` and `secret` to the driver directly. Do not commit them to your repository. ## Configuration Point Mobilewright at TestingBot by passing a `TestingBotDriver` instance in your `mobilewright.config.ts`. Because TestingBot sessions start with the app already installed, the app under test has to be declared up front in the `apps` option. ```typescript import { defineConfig } from 'mobilewright'; import { TestingBotDriver } from '@testingbot/mobilewright-driver'; export default defineConfig({ testDir: '.', bundleId: 'com.example.MyApp', driver: new TestingBotDriver({ apps: { android: './build/app.apk', 'ios-simulator': './build/app-sim.zip', 'ios-real': './build/app.ipa', // must be test-signed for real devices }, }), projects: [ { name: 'android', use: { platform: 'android', deviceType: 'emulator' } }, { name: 'ios', use: { platform: 'ios', deviceType: 'real', osVersion: '>=17' } }, ], }); ``` A few things worth knowing about `apps`: - Local `.apk`, `.ipa` and simulator `.zip` paths are uploaded to [TestingBot Storage](https://testingbot.com/support/api/storage#upload) automatically. Already-uploaded builds can be referenced with their `tb://` URL. - The keys are `android`, `android-emulator`, `android-real`, `ios`, `ios-simulator` and `ios-real`. The most specific one wins: `ios-real` overrides `ios`. - A value can be an array. The first entry is the app under test, the rest are installed alongside it, which is how you ship a mock server or a test harness with your build. A Mobilewright test looks the same whether it runs locally or on TestingBot: ```typescript import { test, expect } from '@mobilewright/test'; test('shows the welcome screen', async ({ device, screen, bundleId }) => { await device.launchApp(bundleId); await expect(screen.getByText('Welcome')).toBeVisible(); }); ``` ## Selecting devices Device selection comes from the Mobilewright project config, in the `use` block. Regular expressions and version ranges work on both real and virtual devices. | Key | Example | Notes | | --- | --- | --- | | `platform` | `'ios'` or `'android'` | Required. | | `deviceType` | `'real'`, `'simulator'`, `'emulator'` | Defaults to a virtual device. | | `deviceName` | `/iPhone 1[45]/`, `/pixel/i` | Regex, matched against the TestingBot device catalogs. | | `osVersion` | `'17'`, `'17.4'`, `'>=16 <18'` | Exact version, prefix or range expression. | | `deviceId` | `'2241'` | Pin one specific physical device. The id comes from [GET /v1/devices](https://testingbot.com/support/api/devices). The Mobilewright docs mark `deviceId` as local-driver-only; this driver resolves it against the TestingBot catalog, so it works here too. | Real devices resolve against the live physical catalog, where idle devices are preferred. If the device you asked for is busy, TestingBot queues your session until it frees up rather than failing. Virtual devices resolve against the simulator and emulator catalog, where the newest matching OS version wins. See the [supported device list](https://testingbot.com/support/app-automate/devices) for what is currently available. ## Running tests ```bash npx mobilewright test ``` Each device slot maps to a TestingBot session on your [dashboard](https://testingbot.com/members), with video and a pass or fail verdict reported automatically. Prefix the command with `DEBUG=testingbot:*` to print every allocation, hub command, upload and result the driver performs. ```bash DEBUG=testingbot:* npx mobilewright test ``` ## Driver options Everything the driver accepts. Only `apps` is required; the values below are examples, and the comments call out the real defaults. Anything left unset is simply not sent, in which case the TestingBot server-side default applies. ```typescript new TestingBotDriver({ apps: { android: './build/app.apk' }, // app under test, most specific key wins // ('ios-real' beats 'ios'); tb:// URLs allowed. // Array = app under test first, helper apps after: // android: ['./app.apk', './mock-server.apk'] key: '...', // default: TESTINGBOT_KEY (or TB_KEY) env secret: '...', // default: TESTINGBOT_SECRET (or TB_SECRET) env sessionPerTest: false, // true = fresh session (and video) per test name: 'checkout flow', // session name (default 'mobilewright') hubUrl: '...', // default: https://hub.testingbot.com/wd/hub apiUrl: '...', // default: https://api.testingbot.com/v1 build: 'ci-1234', // default: auto-detected from CI env / TESTINGBOT_BUILD testResults: 'on', // 'off' disables pass/fail reporting allocationTimeout: 395_000, // ms to wait for a device (default 395_000) commandTimeout: 60_000, // ms per WebDriver command (default 60_000) idleTimeout: 230, // seconds before TestingBot reaps an idle session (default 230) maxDuration: 1800, // max session length in seconds (unset by default) appiumVersion: '2.11.2', // pin the Appium version (unset = TestingBot's default) timeZone: 'Europe/Brussels', // device timezone (tz database name) geoCountryCode: 'DE', // route device traffic via a proxy in this country throttleNetwork: '3G', // or { downloadSpeed, uploadSpeed, latency, loss } autoGrantPermissions: true, // Android: auto-grant app permission dialogs autoAcceptAlerts: true, // iOS: auto-accept system permission alerts screenshots: true, // screenshot at every step (default false) video: true, // session video (default true) recordLogs: 'strip-parameters', // command logs: true | false | 'strip-parameters' public: false, // make results publicly accessible extra: 'commit=abc123', // custom metadata on the test detail page tabletOnly: false, // restrict allocation to tablets phoneOnly: false, // ...or to phones capabilities: {}, // extra Appium capabilities (escape hatch) tbOptions: {}, // extra tb:options entries (escape hatch) }); ``` `capabilities` and `tbOptions` are the escape hatches for anything the driver does not expose directly. See the [Appium capabilities reference](https://testingbot.com/support/app-automate/appium/capabilities) for what you can put in them. ## TestingBot commands inside tests Some TestingBot features are runtime commands rather than session capabilities. Import the `testingbot` helper and call it from a test body; it targets the device session Mobilewright connected for that test. ```typescript import { test, expect } from '@mobilewright/test'; import { testingbot } from '@testingbot/mobilewright-driver'; test('checkout survives a slow network', async ({ device, screen, bundleId }) => { await device.launchApp(bundleId); await testingbot.annotate('starting checkout'); // shows in the session timeline await testingbot.throttle('3G'); await expect(screen.getByText('Order placed')).toBeVisible(); console.log(testingbot.dashboardUrl()); }); ``` | Command | TestingBot command | What it does | | --- | --- | --- | | `throttle(conditions)` | `tb:throttle` | Network conditions mid-test: `'Edge'`, `'3G'`, `'4G'`, `'airplane'`, `'disable'`, or `{ downloadSpeed, uploadSpeed, latency, loss }` in kb/s and ms. | | `annotate(text)` | `tb:test-context` | Logs a step into the session command list so the timeline shows what the test was doing. | | `setName(name)` | `tb:test-name` | Names the session on the dashboard. | | `setBuild(build)` | `tb:test-build` | Groups the session under a build. | | `setTags([...])` | `tb:test-tags` | Tags the session. | | `setResult(passed)` | `tb:test-result` | Overrides the verdict, as a boolean or `'passed'` / `'failed'`. The driver reports it automatically, so use this only for outcomes Mobilewright cannot see. | | `updateInfo({...})` | `tb:test-info` | Bulk update of `name`, `build`, `public`, `statusMessage` and `extra`. | | `breakpoint()` | `tb:break` | Pauses the session for manual inspection with a live view. Debugging only, never leave it in CI. | | `shell(command, args)` | `mobile: shell` | ADB shell on Android. A whitelisted subset is available on physical devices. | | `execute(script, ...args)` | any | Escape hatch for any Appium or TestingBot execute-script command. | | `sessionId()` / `dashboardUrl()` | - | This test's TestingBot session id and dashboard link. | Throttling is reset automatically when the test's device session is released, so slow-network conditions never leak into the next test that reuses the device. Calling a command outside a test body raises an explanatory error rather than a null reference. ### Recipe: name every session after its test ```typescript test.beforeEach(async ({ device }, testInfo) => { await testingbot.setName(testInfo.title); await testingbot.setTags([testInfo.project.name]); }); ``` Requesting the `device` fixture is what makes the session exist before the hook runs. With `sessionPerTest: true` each session gets exactly one name; in pooled mode the last test to run on a session wins. ## Sessions, naming and reporting - By default a device slot's session is reused across tests, so you pay device startup once. A session that hosted exactly one test is renamed after that test and carries its own verdict. A session that hosted several keeps its configured name (`mobilewright` unless you set one) and gets a run summary in its **status message** instead, such as `12/12 tests passed`, or `1 of 3 tests failed. ...` with the individual failure messages appended. - Set `sessionPerTest: true` for one session, one video and one named verdict per test. That is the clearest dashboard, at the cost of device startup on every test. - Every session is tagged `mobilewright`, so you can filter the dashboard down to these runs. Git metadata is reported on top of that when Mobilewright captures it: the commit hash, branch, author and subject land in the session's **extra** field, alongside your own `extra` rather than replacing it, and the branch is added as a second tag. Enable it with `captureGitInfo: { commit: true }` in `mobilewright.config.ts`. - Failure messages are reported to the session's status message. `build` groups sessions on the dashboard and is auto-detected from GitHub Actions, GitLab, CircleCI, Buildkite, Bitrise, Travis, Azure DevOps, Jenkins and TeamCity, or read from a `TESTINGBOT_BUILD` variable. ## Testing localhost and staging servers If your app talks to a backend that is not reachable from the internet, use the [TestingBot Tunnel](https://testingbot.com/support/tunnel). Either run the tunnel yourself and hand the driver its identifier: ```typescript driver: new TestingBotDriver({ tunnelIdentifier: 'my-tunnel' }) ``` Or let the driver manage the tunnel around the run. This requires Java and `npm install --save-dev testingbot-tunnel-launcher`: ```typescript driver: new TestingBotDriver({ tunnel: true }) // anonymous tunnel driver: new TestingBotDriver({ tunnel: { identifier: 'ci-42' } }) // named tunnel ``` The driver starts the tunnel before any device is allocated and closes it after the run. ## Running in CI/CD Store your credentials as secrets and run Mobilewright like any other test command. The build name is auto-detected, so no extra configuration is needed. ```yaml jobs: mobile-tests: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: { node-version: 20, cache: npm } - run: npm ci - run: npx mobilewright test env: TESTINGBOT_KEY: ${{ secrets.TESTINGBOT_KEY }} TESTINGBOT_SECRET: ${{ secrets.TESTINGBOT_SECRET }} # build name auto-detects as "owner/repo #run" - no config needed ``` The same pattern works on the other CI systems listed under [CI/CD integrations](https://testingbot.com/support/integrations/ci-cd). ## How Mobilewright maps to TestingBot | Mobilewright concept | TestingBot implementation | | --- | --- | | Device allocation | An Appium session on `hub.testingbot.com`. The session UUID is the device id, and busy devices queue server side. | | App install | Uploaded to TestingBot Storage at allocation time and passed as `appium:app`, helper apps as `appium:otherApps`. Real iOS builds are re-signed automatically. There are no mid-session installs. | | Taps, swipes and gestures | W3C pointer actions. | | View hierarchy | Appium page source, mapped to Mobilewright's `ViewNode` tree. | | Webviews | Appium contexts, through `webViewBridge`. | | Pass and fail reporting | `PUT /v1/tests/:session` from the `TestObserver` hooks. | | Video | Recorded by TestingBot. `startRecording({ output })` downloads the MP4 locally at run end. | ## Troubleshooting | Symptom | Cause and fix | | --- | --- | | `TestingBot credentials are missing` | Set `TESTINGBOT_KEY` and `TESTINGBOT_SECRET`, or pass `{ key, secret }` to the driver. | | `TestingBot sessions must start with an app` | Declare the app in the driver's `apps` option, per platform and device type. | | `No TestingBot real device matches ...` | Your criteria matched nothing in the catalog. Check the [available devices](https://testingbot.com/support/app-automate/devices) and loosen `osVersion` or `deviceName`. | | Allocation waits for minutes | The matching device is busy. TestingBot queues the session until it frees up, bounded by `allocationTimeout`. | | `Test timeout ... while setting up "device"` | More `workers` than your plan's parallel limit. TestingBot queues up to twice the limit server side and the driver waits out the rest. The driver logs your plan's number: set `workers` to at most that, or raise the Mobilewright `timeout`. | | `... is a simulator-only build and cannot be installed on a real device` | Build a test-signed `.ipa` for `deviceType: 'real'` projects. | | `"..." is not installed on this device` | The config's `bundleId` is not the package id of the build in `apps`. Check it with `aapt2 dump packagename app.apk`, and look for a stale `bundleId` or environment override. | | `cannot install "..." mid-session` | TestingBot has no mid-session installs. Every app in `installApps` must also be listed in the driver's `apps` option. | | Sessions show a run summary instead of a test name | The session hosted several tests, so the summary goes to the status message and the name is left alone. Use `sessionPerTest: true` to get one named session per test. | ## Current limitations - One TestingBot session may host several Mobilewright tests unless `sessionPerTest: true`. Multi-test sessions report the run level verdict. - Modifier key chords such as `pressKeys(['ctrl+a'])` work on Android only, because XCUITest cannot hold modifier keys. - `pressButton` on iOS supports `HOME`, `VOLUME_UP` and `VOLUME_DOWN`. `listApps()` reports the foreground app only. - Screenshots are always PNG. `applyDeviceSettings` turns Android animations off on a best-effort basis and is a no-op on iOS. - Real iOS devices need a test-signed `.ipa`. Simulator builds are rejected on real devices with a clear error. - While attached to a webview, the session's single Appium context is the web layer. ## Frequently asked questions What is Mobilewright? Mobilewright is an open source mobile UI test framework from the mobile-next project. You write tests in TypeScript or JavaScript with a Playwright-style API (`device`, `screen`, `expect`) and run them against iOS and Android apps. TestingBot publishes `@testingbot/mobilewright-driver`, a driver that points those same tests at our device cloud. How do I run Mobilewright tests on TestingBot? Run `npx @testingbot/mobilewright-driver init` in your test project to scaffold a config and a sample test, export `TESTINGBOT_KEY` and `TESTINGBOT_SECRET`, then run `npx mobilewright test`. See the [quickstart](https://testingbot.com#quickstart) for the manual setup. Can I run Mobilewright on real iOS and Android devices? Yes. Set `deviceType: 'real'` in a project's `use` block to allocate physical iPhones, iPads and Android phones or tablets from our EU datacenter. Leave it out and you get an iOS simulator or Android emulator instead. Real iOS devices need a test-signed `.ipa`. Do I have to upload my app before running? No. Declare your builds in the driver's `apps` option and local `.apk`, `.ipa` and simulator `.zip` files are uploaded to [TestingBot Storage](https://testingbot.com/support/api/storage#upload) for you. If you already uploaded a build you can reference its `tb://` URL instead. Why do I get "Test timeout while setting up device"? You are running more Mobilewright `workers` than your plan's parallel limit. TestingBot queues up to twice the limit server side and the driver waits out the rest, so the test-scoped device fixture eventually times out. The driver logs your plan's parallel limit: set `workers` to at most that number, or raise the Mobilewright `timeout`. Can I test a staging or localhost backend? Yes. Pass `tunnelIdentifier` to reuse a [TestingBot Tunnel](https://testingbot.com/support/tunnel) you started yourself, or let the driver open and close one around the run with `tunnel: true`. See [Testing localhost and staging](https://testingbot.com#tunnel). Why does one session contain several tests? By default a device slot's session is reused across tests so you only pay device startup once, and such a session reports a run level verdict in its status message, such as "12/12 tests passed". Set `sessionPerTest: true` for one session, one video and one named verdict per test. How do I debug a failing Mobilewright run? Prefix the run with `DEBUG=testingbot:*` to log every allocation, hub command, upload and result the driver performs. Every session also has video, command logs and device logs on its test detail page in the [dashboard](https://testingbot.com/members). ## Next steps - [ **Driver on GitHub** Source, issues and releases](https://github.com/testingbot/mobilewright-testingbot) - [ **Supported devices** Real iPhones, iPads and Android devices on the grid](https://testingbot.com/support/app-automate/devices) - [ **Real device testing** How the real-hardware cloud works](https://testingbot.com/real-device-testing) - [ **TestingBot Tunnel** Test against internal or staging APIs](https://testingbot.com/support/tunnel) - [ **Appium guide** The protocol the driver speaks underneath](https://testingbot.com/support/app-automate/appium) - [ **Pricing** Real-device plans and concurrency](https://testingbot.com/pricing) ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/espresso # Espresso Testing TestingBot provides access to Android physical devices and Android emulators, capable of running your Android Espresso tests. [Espresso](https://developer.android.com/training/testing/espresso) is a mobile automation framework from Google, designed to run UI tests for Android applications. Using Espresso in combination with TestingBot's Android cloud service is very straightforward: - Upload your native mobile Android app, together with the Espresso test apk file. These are created with either Java or Kotlin. - Specify on which devices you want to run the Espresso tests. - Retrieve the results of the Espresso tests. ## Getting Started To get started, please follow the steps below: 1. You will need a `key` and `secret` to connect to the TestingBot grid. These credentials are available in the [TestingBot member area](https://testingbot.com/members). 2. Espresso tests require a separate `apk` file for testing. So you will need to upload both the app (`.apk` or `.aab` file) and the test file (`.apk`). You can also use the [TestingBot Espresso Demo App](https://github.com/testingbot/android-espresso-demo-app) to give the Espresso Cloud Testing functionality a try. ## TestingBot CLI The easiest way to run Espresso tests on TestingBot is using our CLI tool. It handles uploading your app, test APK, and running tests in a single command. ### Installation ```bash npm install -g @testingbot/cli ``` ### Authentication Authenticate using this command: ```bash testingbot login ``` This opens your browser for authentication. After logging in, your credentials are saved to `~/.testingbot` and you can start using the CLI. #### Other Authentication Methods - **Command-line options** : `--api-key` and `--api-secret` - **Environment variables** : `TB_KEY` and `TB_SECRET` - **Config file** : Create `~/.testingbot` with content `key:secret` ### Quick Start Run Espresso tests with a single command: ```bash testingbot espresso app.apk app-test.apk --device "Pixel 8" --platform-version "14" ``` ```bash testingbot espresso app.apk app-test.apk --device "Galaxy S24" --real-device ``` The CLI will: 1. Upload your app APK to TestingBot 2. Upload your test APK 3. Start the test run 4. Show real-time progress and results If you build your Android app with Gradle, our [Gradle plugin](https://testingbot.com/support/integrations/ci-cd/gradle) can upload your app and run Espresso tests straight from your build, no separate CLI step required. ## Upload Android App To run Android Espresso tests on TestingBot, you will first have to upload your `.apk` or `.aab` file to TestingBot Storage. [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) When using the CLI, the app upload is handled automatically as part of the `testingbot espresso` command. Simply specify your app APK file as the first argument: ```bash testingbot espresso /path/to/app/Application-debug.apk /path/to/test/Application-debug-test.apk --device "Pixel 8" ``` Alternatively, you can use the `--app` option: ```bash testingbot espresso --app /path/to/app/Application-debug.apk --test-app /path/to/test/Application-debug-test.apk --device "Pixel 8" ``` ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/espresso/app" \ -F "file=@/path/to/app/file/Application-debug.apk" ``` The API response will include an `id` response, which you can use in the other API calls. ```json { "id": 4012 } ``` ## Upload Espresso Test Suite You can now upload the `.apk` file containing the Espresso tests. The upload process is similar to the [app upload](https://testingbot.com#upload) process. [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) When using the CLI, the test APK upload is handled automatically. Specify your test APK as the second argument: ```bash testingbot espresso app.apk app-test.apk --device "Pixel 8" ``` Or use the `--test-app` option: ```bash testingbot espresso --app app.apk --test-app app-test.apk --device "Pixel 8" ``` ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/tests" \ -F "file=@/path/to/app/file/Application-debug-test.apk" ``` Replace the `:id` with the identifier you received during the app upload call. ## Specify devices You can specify one or more Android device capabilities with your API call. These capabilities can target either Android emulators, or physical Android devices. ![Android OS selected](https://testingbot.com/assets/environments/svg/android-1b3d235e17522e2238e1c55b4e9cdd41219f653cd958ea065877d3813f32ffc3.svg) Android 16.0 › Pixel 9 Loading environments... Please wait while we load the available browsers and platforms. We offer special parameters which you can use to allocate a device: Regex Input | Result || `"Pixel.*"` | This will allocate any available Pixel device (phone) | | `"*"` | This will allocate a random available Android device | | `"Pixel [8-9]"` | This will allocate either an Pixel 8 or 9 | Some Examples: ```bash # find any Samsung Galaxy, except the S23 family -d '{"capabilities":[{"deviceName":"^Galaxy(?!.*23).*$", "platformName":"Android", "realDevice": true}]}' # find any device which name starts with Pixel -d '{"capabilities":[{"deviceName":"Pixel.*", "platformName":"Android", "realDevice": true}]}' ``` ### Tablet Only You can specify the `tabletOnly` capability when you only want to allocate a tablet device. ### Phone Only You can specify the `phoneOnly` capability when you only want to allocate a phone device. ```bash -d '{"capabilities":[{"deviceName":"*", "tabletOnly": true, "platformName":"Android", "realDevice": true}]}' -d '{"capabilities":[{"deviceName":"*", "phoneOnly": true, "platformName":"Android", "realDevice": true}]}' ``` ## Run Espresso Tests Now that you have uploaded both your Android application and the Espresso test suite, you can run your first Espresso test on the TestingBot cloud. [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) Use the CLI to run tests with device selection options: ```bash # Basic usage with device selection testingbot espresso app.apk app-test.apk --device "Pixel 8" --platform-version "14" # Real device testingbot espresso app.apk app-test.apk --device "Galaxy S24" --real-device # With test name and build identifier testingbot espresso app.apk app-test.apk \ --device "Pixel 8" \ --platform-version "14" \ --name "Smoke Tests" \ --build "v2.1.0" ``` ### Device Options | Option | Description | | --- | --- | | `--device ` | Device name (e.g., "Pixel 8", "Samsung.\*") | | `--platform-version ` | Android version (e.g., "12", "13", "14") | | `--real-device` | Use a real device instead of emulator | | `--tablet-only` | Only allocate tablet devices | | `--phone-only` | Only allocate phone devices | #### Test Filtering ```bash # Run specific test classes testingbot espresso app.apk app-test.apk \ --device "Pixel 8" \ --class "com.example.LoginTest,com.example.HomeTest" # Run tests with specific annotations testingbot espresso app.apk app-test.apk \ --device "Pixel 8" \ --annotation "com.example.SmokeTest" \ --size "small,medium" # Exclude specific test classes testingbot espresso app.apk app-test.apk \ --device "Pixel 8" \ --not-class "com.example.SlowTest" ``` #### Test Filtering Options | Option | Description | | --- | --- | | `--class ` | Run tests in specific classes (comma-separated fully qualified names) | | `--not-class ` | Exclude tests in specific classes | | `--package ` | Run tests in specific packages (comma-separated) | | `--not-package ` | Exclude tests in specific packages | | `--annotation ` | Run tests with specific annotations (comma-separated) | | `--not-annotation ` | Exclude tests with specific annotations | | `--size ` | Run tests by size: small, medium, large (comma-separated) | #### Network & Location ```bash # With network throttling and geolocation testingbot espresso app.apk app-test.apk \ --device "Pixel 8" \ --throttle-network "3G" \ --geo-location "DE" \ --language "de" ``` ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \ -d '{"capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \ -H "Content-Type: application/json" ``` Replace the `:id` with the identifier you received during the app upload call. The API response will include a `success` response, indicating if the tests will start or if an error occurred. ```json { "success": true } ``` ## View Espresso Test Results You will now see a test appear in the [TestingBot dashboard](https://testingbot.com/members), together with a video, test logs and other metadata. You can also use the CLI or an API call to get the results from the run(s): [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) By default, the CLI waits for test completion and displays real-time progress: ```bash testingbot espresso app.apk app-test.apk --device "Pixel 8" ``` The CLI will show: - Upload progress for app and test APK - Device allocation status - Live output from Espresso tests - Final pass/fail status ### Async Mode Use `--async` to start tests without waiting for results: ```bash testingbot espresso app.apk app-test.apk --device "Pixel 8" --async ``` #### Download JUnit Report Download the JUnit report after test completion: ```bash testingbot espresso app.apk app-test.apk --device "Pixel 8" \ --report junit \ --report-output-dir ./reports ``` ```bash curl -u api_key:api_secret \ "https://api.testingbot.com/v1/app-automate/espresso/:id" ``` Replace the `:id` with the identifier you received during the app upload call. The API response will include a response indicating the current state of the Espresso test results. ```json { "runs": [ { "status": "DONE", "capabilities": { "version": "12", "deviceName": "Pixel 6", "platformName": "Android" }, "success": false, "test": { "sessionId": "30f642d082c3-804f3c0ec2df-a18b2d431ffb-169920992163-48455547", "environment": { "name": "chrome", "os": "Pixel 6 - 12.0", "version": "12.0" } } } ], "version": "1.0" } ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/espresso/instrumentation # Espresso Instrumentation Logs Instrumentation logs are logs generated by Android Espresso. These logs list all the steps that were performed during the test, with any errors or meta data available from the Android Espresso test runner (usually `AndroidJUnitRunner`). These logs are available in the [TestingBot dashboard](https://testingbot.com/members) as well as through the [REST API](https://testingbot.com/support/api). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/espresso/filter # Filter Espresso Test Cases By default, TestingBot will execute all tests present in the Espresso `.apk`. The Espresso test runner (which is usually **AndroidJUnitRunner** ) provides multiple options to filter test cases for execution. You might want to use these filter options for specific use cases, such as: - Run only specific tests for a bugfix. - Only run a subset of tests, for quick smoke testing. Instead of waiting on all your Espresso tests. - Run specific tests for specific environments (production, staging, ...) Below you'll find the different filter options that TestingBot and Espresso provide. - [class](https://testingbot.com#class): will run all tests in a specific class - [notClass](https://testingbot.com#notClass): will run all tests except the tests in a particular class - [size](https://testingbot.com#size): will run a specific test size annotated with `SmallTest`, `MediumTest` or `LargeTest` - [package](https://testingbot.com#package): will run all tests in a specific Java package - [notPackage](https://testingbot.com#notPackage): will run all tests except the tests in a specific package - [annotation](https://testingbot.com#annotation): will run all tests with a specific annotation - [notAnnotation](https://testingbot.com#notAnnotation): will run all tests except the tests with a specific annotation - [numShards](https://testingbot.com#shards): filter test run to a shard of all tests, where numShards is an integer greater than 0 and shardIndex is an integer between 0 (inclusive) and numShards (exclusive) ## class Filter To only run Espresso tests for a specific `class`, you can use the class option. You can pass a list of fully qualified Java class names. See the example below. [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash testingbot espresso app.apk app-test.apk \ --device "Pixel 6" \ --platform-version "12" \ --class "com.android.app.one,com.android.app.two" ``` This will start an Espresso test session on the specified device and only run tests for the classes specified. ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \ -d '{ "espressoOptions": { "class": ["com.android.app.one", "com.android.app.two"] }, "capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \ -H "Content-Type: application/json" ``` This call with start an Espresso test Session on the specific device and only run the Espresso tests for the classNames specified. ## notClass Filter Use the `notClass` option to run all Espresso tests except for the tests in the classes specified with this option. You can pass a list of fully qualified Java class names. See the example below. [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash testingbot espresso app.apk app-test.apk \ --device "Pixel 6" \ --platform-version "12" \ --not-class "com.android.app.one,com.android.app.two" ``` ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \ -d '{ "espressoOptions": { "notClass": ["com.android.app.one", "com.android.app.two"] }, "capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \ -H "Content-Type: application/json" ``` ## size Filter The `size` option allows you to specify which tests you want to run, depending on the annotated size: `SmallTest`, `MediumTest` or `LargeTest` You can pass an array of sizes, see below for an example. [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash testingbot espresso app.apk app-test.apk \ --device "Pixel 6" \ --platform-version "12" \ --size "small,medium" ``` ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \ -d '{ "espressoOptions": { "size": ["small", "medium"] }, "capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \ -H "Content-Type: application/json" ``` ## package Filter The `package` filter allows you to specify for which package the Espresso tests should run. You can pass an array of packages, see below for an example. [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash testingbot espresso app.apk app-test.apk \ --device "Pixel 6" \ --platform-version "12" \ --package "com.android.one,com.android.two" ``` ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \ -d '{ "espressoOptions": { "package": ["com.android.one", "com.android.two"] }, "capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \ -H "Content-Type: application/json" ``` ## notPackage Filter The `notPackage` filter allows you to indicate that Espresso tests should run for everything except for the packages defined. You can pass an array of packages, see below for an example. [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash testingbot espresso app.apk app-test.apk \ --device "Pixel 6" \ --platform-version "12" \ --not-package "com.android.one,com.android.two" ``` ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \ -d '{ "espressoOptions": { "notPackage": ["com.android.one", "com.android.two"] }, "capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \ -H "Content-Type: application/json" ``` ## annotation Filter The `annotation` filter will run tests annotated with the field (or fields) you specified. You can pass an array of annotations, see below for an example. [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash testingbot espresso app.apk app-test.apk \ --device "Pixel 6" \ --platform-version "12" \ --annotation "com.android.foo.MyAnnotation,com.android.foo.MyAnnotationTwo" ``` ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \ -d '{ "espressoOptions": { "annotation": ["com.android.foo.MyAnnotation", "com.android.foo.MyAnnotationTwo"] }, "capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \ -H "Content-Type: application/json" ``` ## notAnnotation Filter The `notAnnotation` filter will run all tests except those annotated with the field (or fields) you specified. You can pass an array of notAnnotations, see below for an example. [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash testingbot espresso app.apk app-test.apk \ --device "Pixel 6" \ --platform-version "12" \ --not-annotation "com.android.foo.MyAnnotation,com.android.foo.MyAnnotationTwo" ``` ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \ -d '{ "espressoOptions": { "notAnnotation": ["com.android.foo.MyAnnotation", "com.android.foo.MyAnnotationTwo"] }, "capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \ -H "Content-Type: application/json" ``` ## shards Filter You can split your Espresso tests across multiple parallel shards to reduce total run time. Pass `numShards` (total number of shards, an integer greater than 0) and `shardIndex` (which shard this run executes, an integer between 0 inclusive and `numShards` exclusive). For example, to split into 4 shards and run shard 0 (the first), pass `numShards=4` and `shardIndex=0`. Submit one test run per shard index to run all four shards in parallel. [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash testingbot espresso app.apk app-test.apk \ --device "Pixel 6" \ --platform-version "12" \ --num-shards 4 \ --shard-index 0 ``` ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \ -d '{ "espressoOptions": { "numShards": 4, "shardIndex": 0 }, "capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \ -H "Content-Type: application/json" ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/espresso/options # Set Espresso Test Runner By default, TestingBot will run your Espresso tests with `${packageName}/androidx.test.runner.AndroidJUnitRunner` (TestingBot substitutes `${packageName}` with your app's package id automatically). If you have defined your own `testInstrumentationRunner`, you can pass this custom runner during the start Espresso call: [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash testingbot espresso app.apk app-test.apk \ --device "Pixel 6" \ --platform-version "12" \ --test-runner "${packageName}/customTestRunner" ``` ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \ -d '{ "espressoOptions": { "testRunner": "${packageName}/customTestRunner" }, "capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \ -H "Content-Type: application/json" ``` Replace the `:id` with the identifier you received during the app upload call. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/espresso/screenshots # Capturing Espresso Screenshots Espresso does not have a built-in way to capture screenshots during test automation. You can however achieve this by using two options: - **Native Screenshot grabbing** Use native Java or Kotlin code to take a screenshot of the device, and save it on the device. Once the test is completed, you can fetch the image through `ADB`, or if you run it via TestingBot, the screenshot will appear on the [test result page](https://testingbot.com#view). - **Using the Spoon library** The [Square Screenshots feature](https://square.github.io/spoon/#screenshots) is part of the Spoon Library. It will take screenshots during your Espresso tests, but only for Android devices with a version of Android 10 or lower. [Using Native Screenshot](https://testingbot.com#) [Using Spoon Library](https://testingbot.com#) 1. **Add the native screenshots file to your project** You can see the implementation of Native Screenshots in the [TestingBot Espresso Demo App](https://github.com/testingbot/android-espresso-demo-app/blob/main/app/src/androidTest/java/com/testingbot/calculator/NativeScreenshot.java). ```java package com.testingbot.calculator; import android.graphics.Bitmap; import android.os.Build; import android.os.Environment; import androidx.test.runner.screenshot.BasicScreenCaptureProcessor; import androidx.test.runner.screenshot.ScreenCapture; import androidx.test.runner.screenshot.Screenshot; import java.io.File; import java.io.IOException; import java.util.regex.Pattern; public final class NativeScreenshot { private static String methodName; private static String className; private static final Pattern SCREENSHOT_NAME_VALIDATION = Pattern.compile("[a-zA-Z0-9_-]+"); private NativeScreenshot() {} /** * Captures screenshot using Androidx Screenshot library and stores in the filesystem. * If there is any runtime exception while capturing screenshot, the method throws * Exception and the test might fail if exception is not handled properly. * @param screenshotName a screenshot identifier * @return path to the screenshot file */ public static String capture(String screenshotName) { StackTraceElement testClass = findTestClassTraceElement(Thread.currentThread().getStackTrace()); className = testClass.getClassName().replaceAll("[^A-Za-z0-9._-]", "_"); methodName = testClass.getMethodName(); EspressoScreenCaptureProcessor screenCaptureProcessor = new EspressoScreenCaptureProcessor(); if (!SCREENSHOT_NAME_VALIDATION.matcher(screenshotName).matches()) { throw new IllegalArgumentException("ScreenshotName must match " + SCREENSHOT_NAME_VALIDATION.pattern() + "."); } else { ScreenCapture capture = Screenshot.capture(); capture.setFormat(Bitmap.CompressFormat.PNG); capture.setName(screenshotName); try { return screenCaptureProcessor.process(capture); } catch (IOException e) { throw new RuntimeException("Unable to capture screenshot.", e); } } } /** * Extracts the currently executing test's trace element based on the test runner * or any framework being used. * @param trace stacktrace of the currently running test * @return StackTrace Element corresponding to the current test being executed. */ private static StackTraceElement findTestClassTraceElement(StackTraceElement[] trace) { for(int i = trace.length - 1; i >= 0; --i) { StackTraceElement element = trace[i]; if ("android.test.InstrumentationTestCase".equals(element.getClassName()) && "runMethod".equals(element.getMethodName())) { return extractStackElement(trace, i); } if ("org.junit.runners.model.FrameworkMethod$1".equals(element.getClassName()) && "runReflectiveCall".equals(element.getMethodName())) { return extractStackElement(trace, i); } if ("cucumber.runtime.model.CucumberFeature".equals(element.getClassName()) && "run".equals(element.getMethodName())) { return extractStackElement(trace, i); } } throw new IllegalArgumentException("Could not find test class!"); } /** * Based on the test runner or framework being used, extracts the exact traceElement. * @param trace stacktrace of the currently running test * @param i a reference index * @return trace element based on the index passed */ private static StackTraceElement extractStackElement(StackTraceElement[] trace, int i) { int testClassTraceIndex = Build.VERSION.SDK_INT >= 23 ? i - 2 : i - 3; return trace[testClassTraceIndex]; } private static class EspressoScreenCaptureProcessor extends BasicScreenCaptureProcessor { private static final String SCREENSHOT = "screenshots"; EspressoScreenCaptureProcessor() { File screenshotDir = new File(String.valueOf(Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_DOWNLOADS)), SCREENSHOT); File classDir = new File(screenshotDir, className); mDefaultScreenshotPath = new File(classDir, methodName); } /** * Converts the filename to a standard path to be stored on device. * Example: "post_addition" converts to "1648038895211_post_addition" * which is later suffixed by the file extension i.e. png. * @param filename a screenshot identifier * @return custom filename format */ @Override protected String getFilename(String filename) { return System.currentTimeMillis() + "_" + filename; } } } ``` 2. **Modify your Espresso test scripts to take screenshots** You can call the `NativeScreenshot.capture("tag")` method to capture screenshots in your Espresso tests. The `tag` can be used to provide a name to the screenshot (usually you would use the testname for this). ```java @Test public void ensureCalculatorWorks() { onView(withId(R.id.buttonOne)).perform(click()); onView(withId(R.id.buttonAdd)).perform(click()); onView(withId(R.id.buttonTwo)).perform(click()); onView(withId(R.id.editText)).check(matches(withText("3"))); NativeScreenshot.capture("ensureCalculatorWorks"); } ``` 3. **Android 11 and higher** For Android 11 and higher, make sure to add the following `RuleChain` to your Espresso test: ```java @Rule public RuleChain screenshotRule = RuleChain .outerRule(GrantPermissionRule.grant(Manifest.permission.WRITE_EXTERNAL_STORAGE, Manifest.permission.READ_EXTERNAL_STORAGE)) .around(activityScenarioRule); ``` 1. **Add the required dependencies to your project** See the [Spoon Documentation](https://github.com/square/spoon#download) on how to add Spoon to your project. You can either use a `JAR` or use `Maven`. 2. **Change storage permissions** If you do not yet have the `WRITE_EXTERNAL_STORAGE` permission in your `AndroidManifest.xml`, please add it. It will allow Spoon to save the screenshots on your device. ```xml ... ``` If the targetSdk for your app is using Android 10 (level 29), please make sure to add `android:requestLegacyExternalStorage="true"` in the Application tag of your `AndroidManifest.xml` file. 3. **Modify your Espresso test scripts to take screenshots** You can call the `Spoon.screenshot(activityName, tag)` method to capture screenshots in your Espresso tests. The `tag` can be used to provide a name to the screenshot (usually you would use the testname for this). ```java @Rule public ActivityScenarioRule activityRule = new ActivityScenarioRule<>(MainActivity.class); @Test public void ensureCalculatorWorks() { onView(withId(R.id.buttonOne)).perform(click()); onView(withId(R.id.buttonAdd)).perform(click()); onView(withId(R.id.buttonTwo)).perform(click()); onView(withId(R.id.editText)).check(matches(withText("3"))); activityRule.getScenario().onActivity(activity -> Spoon.screenshot(activity, "ensureCalculatorWorks")); } ``` ## Viewing Espresso Screenshots The Espresso tests that you run on TestingBot will appear automatically in the TestingBot dashboard. If screenshots were generated during the test, they will be added on the Espresso test result page. Additionally, these screenshots will also be available for download through our [REST API](https://testingbot.com/support/api). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/espresso/set-ip-geolocation # Use IP geolocation TestingBot's GeoIP feature allows you to test your app from various parts of the world, by using IP addresses in [over 20 different countries](https://testingbot.com/support/web-automate/selenium/test-options#geo). When you enable GeoIP, all traffic from your mobile Android app will originate from a specific IP address hosted in the country that you choose. ## GeoIP example To use GeoIP, you can specify the geoIP option when starting an Espresso test. Please see the example below where we specify that the app should connect via Germany. [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash testingbot espresso app.apk app-test.apk \ --device "Pixel 6" \ --platform-version "12" \ --geo-location "DE" ``` The `DE` abbreviation is the ISO code of Germany. You can find other [GeoIP country codes](https://testingbot.com/support/web-automate/selenium/test-options#geo) to use. ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \ -d '{ "espressoOptions": { "geoLocation": "DE" }, "capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \ -H "Content-Type: application/json" ``` The `DE` abbreviation is the ISO code of Germany. You can find other [GeoIP country codes](https://testingbot.com/support/web-automate/selenium/test-options#geo) to use. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/espresso/set-localization-options # Set localization options With TestingBot you can run Android Espresso tests on a localized version of your Android app. You can configure the language, locale and timezone of the remote device before running an Android Espresso test. ## Setting Language for Android Espresso Tests You can test a localized version of your app with Android Espresso, by changing the language of the application under test. You will need to specify a `language` option with the ISO code of the language you want to use. [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash testingbot espresso app.apk app-test.apk \ --device "Pixel 6" \ --platform-version "12" \ --language "fr" ``` Use an [ISO 639-1 language code](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) to set the language of your app. Your app needs to be compiled with support for this language to notice a change. ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \ -d '{ "espressoOptions": { "language": "fr" }, "capabilities":[{"platform":"ANDROID", "version":12, "deviceName":"Pixel 6", "platformName":"Android"}]}' \ -H "Content-Type: application/json" ``` Use an [ISO 639-1 language code](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) to set the language of your app. Your app needs to be compiled with support for this language to notice a change. ## Set a locale for Android Espresso Testing It is possible to set a locale, different than the default device's locale, with the `locale` parameter. Please use the [ISO 3166-1 alpha-2](https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2) country code. For example: `DE` for Germany. [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash testingbot espresso app.apk app-test.apk \ --device "Pixel 6" \ --platform-version "12" \ --locale "DE" ``` ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \ -d '{ "espressoOptions": { "locale": "DE" }, "capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \ -H "Content-Type: application/json" ``` ## Change timezone for Android Espresso It might be necessary during your tests to change the timezone to a different timezone than the default one on the device (UTC). Please use a timezone format from the [list of timezones](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones). [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash testingbot espresso app.apk app-test.apk \ --device "Pixel 6" \ --platform-version "12" \ --timezone "America/New_York" ``` ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \ -d '{ "espressoOptions": { "timeZone": "America/New_York" }, "capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \ -H "Content-Type: application/json" ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/espresso/simulate-network-conditions # Simulate Network Conditions TestingBot allows you to test your mobile apps under various network conditions, including download speed, upload speed, packet loss and latency. ## Simulate network conditions using a predefined network profile TestingBot provides a set of predefined network profiles which you can choose from. Profile Name | Bandwidth down/up (kbps) | Packet Loss (%) | Latency (ms) || Edge | 250/150 | 0 | 300 | | 3G | 400/100 | 0 | 100 | | 4G | 18000/9000 | 0 | 100 | | airplane | 0 | 100 | 0 | [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash testingbot espresso app.apk app-test.apk \ --device "Pixel 6" \ --platform-version "12" \ --throttle-network "3G" ``` ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \ -d '{ "espressoOptions": { "throttle_network": "3G" }, "capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \ -H "Content-Type: application/json" ``` ## Simulate network conditions using a custom network profile You can specify a custom set of parameters, including: - Download rate (kbps) - Upload rate (kbps) - Packet loss (%) - Latency (ms) [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) The CLI supports predefined network profiles. For custom network profiles, use the cURL API directly. ```bash # Predefined profiles: 4G, 3G, Edge, airplane testingbot espresso app.apk app-test.apk \ --device "Pixel 6" \ --platform-version "12" \ --throttle-network "Edge" ``` ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \ -d '{ "espressoOptions": { "throttle_network": { "uploadSpeed": 10240, "downloadSpeed": 10240, "latency": 0, "loss": 0 } }, "capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \ -H "Content-Type: application/json" ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/espresso/test-reports # Espresso Test Reports TestingBot currently offers both test reports from inside the TestingBot member dashboard, or through REST-API where you can use a [JUnit formatted XML report](https://testingbot.com#junit). ## Dashboard Report When you run Espresso Tests with TestingBot, you will find the test results, together with a video of the test, in the [TestingBot member dashboard](https://testingbot.com/members). ## API report You can query the TestingBot API to fetch the result of the Espresso test run(s). [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) By default, the CLI waits for test completion and displays results in real-time: ```bash testingbot espresso app.apk app-test.apk --device "Pixel 6" ``` The CLI will show: - Upload progress for app and test APK - Device allocation status - Live output from Espresso tests - Final pass/fail status Use `--async` to start tests without waiting for results: ```bash testingbot espresso app.apk app-test.apk --device "Pixel 6" --async ``` ```bash curl -u api_key:api_secret \ "https://api.testingbot.com/v1/app-automate/espresso/:id/:run_id" ``` Replace the `:id` and `:run_id` with the identifiers you received during the app upload call and the run call. You will get back a response similar to this one: ```json {"id":86,"created_at":"2023-11-05T18:45:51.000Z","status":"DONE","capabilities":{"version":"12","deviceName":"Pixel 6","platformName":"Android"},"success":false,"test":{"sessionId":"30f642d082c3-804f3c0ec2df-a18b2d431ffb-169920992163-48455547","environment":{"name":"chrome","os":"Pixel 6 - 12.0","version":"12.0"}},"version":"1.0"} ``` ## JUnit XML report You can query our API to fetch a JUnit XML report which you can use a post-build step in your CI/CD environment. **Report Structure** In the XML report, each session maps to a `testsuite` tag. Inside that tag, multiple `testcase` items are available, each mapping to a single Espresso test. If a test failed, a `failure` or `error` tag will be available. [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) Download the JUnit XML report after test completion: ```bash testingbot espresso app.apk app-test.apk \ --device "Pixel 6" \ --report junit \ --report-output-dir ./reports ``` The schema corresponds to the Android Studio's Gradle schema used when running JUnit tests. ```bash curl -u api_key:api_secret \ "https://api.testingbot.com/v1/app-automate/espresso/:id/report" ``` The schema corresponds to the Android Studio's Gradle schema used when running JUnit tests. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/help # Mobile App Help Please see the topic below for help with Automated App testing: ## Help Topics - [Prepare your App](https://testingbot.com/support/app-automate/help/prepare) - [Upload your App](https://testingbot.com/support/app-automate/help/upload) - [Tunnel + Real Devices](https://testingbot.com/support/app-automate/help/tunnel) - [FAQ](https://testingbot.com/support/app-automate/help/faq) ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/help/prepare # Prepare your Mobile App for Automated Testing TestingBot provides Android Emulators and iOS Simulators as well as [Real Devices](https://testingbot.com/support/app-automate/devices) to run tests against your mobile app. Below are the steps necessary to prepare your mobile app for Automated Testing. ## Building your app You do not need to install any code or plugin to run a test. Below are some things that are necessary to successfully run a mobile test: ### For Android: - Please supply the URL to your `.apk` or `.aab` file. **Important:** the .apk file needs to be compiled for the emulator/device you are running your test on (x86 or arm). - Your app needs to have internet permissions - The URL to the `.apk` file must be publicly available, or you can use our [TestingBot Storage](https://testingbot.com/support/app-automate/help/upload) to temporarily host your mobile app. ### For iOS (Real Device): - Please supply the URL to an [.ipa file](https://testingbot.com#ipa). The URL must be publicly available, or you can use our [TestingBot Storage](https://testingbot.com/support/app-automate/help/upload) to temporarily host your mobile app. - By default, we [automatically resign](https://testingbot.com/support/app-automate/help/app-resigning) your [.ipa file](https://testingbot.com#ipa) so that your development builds can run on our devices. If you don't want this, specify `resigningEnabled: false` in your `tb:options` capabilities. - Please make sure your target version matches the version of the simulator you want to test on. ### For iOS (Simulator): - Please supply the URL to a .zip file that contains your .app - The .app needs to be an iOS Simulator build: - Create a Simulator build: ```bash xcodebuild -sdk iphonesimulator -configuration Debug ``` - Zip the .app bundle: ```bash zip -r MyApp.zip MyApp.app ``` - Please make sure your target version matches the version of the simulator you want to test on. ## Building IPA file To build an IPA file, please follow these steps: - Open your project in XCode. - Select **Generic iOS Device** as a project device target. - In the **Product** menu, select **Archive**. - Once archived, click **Export**. - You will be asked what kind of export mode you want, you can pick any of these methods: **Ad Hoc** , **Enterprise** or **Development mode**. - When asked for **Distribution options** , please set these options: - App Thinning: disabled - Uncheck **Rebuild from Bitcode** - Uncheck **Strip Swift Symbols** - Uncheck **Include manifest for over-the-air installation** - Select your **Distribution Certificate** and **Provisioning Profile (either Automatic or Manual)**. - Now your `IPA` file should be generated, export it to a folder on your hard disk. - You can now upload the `IPA` file to a public URL or use our [TestingBot Storage](https://testingbot.com/support/app-automate/help/upload) to temporarily host your app. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/help/upload # Upload your App In order to run tests against your mobile app, our devices need to be able to download your app. There are two ways to configure your tests to do this: - Upload your app to TestingBot Storage: our private upload servers. - Specify an HTTP or HTTPS url to the app you uploaded on a public internet website. ## TestingBot Storage With TestingBot Storage, you can upload your app to our servers. The advantage of this is that your tests will start faster, as the device can download the app from our internal network, instead of the public internet. To get started, simply submit your app to our API: ```bash $ curl -X POST "https://api.testingbot.com/v1/storage" \ -u "key:secret" -F "file=@/path/to/app/file/Application-debug.apk" ``` This call will return a unique identifier for your app (`{"app_url": "tb://..."}`), which you can use in the desired capabilities (see example below). More information regarding this API call and other similar calls (update uploaded file, delete uploaded file, list uploaded files) are available in [our API documentation](https://testingbot.com/support/api/storage#upload). Note: uploads to TestingBot Storage are automatically deleted after 60 days. ```ruby require 'appium_lib' caps = { "platformName" => "iOS", "appium:deviceName" => "iPhone 15", "appium:platformVersion" => "18.0", "appium:app" => "tb://...", # specify the link to your app here "tb:options" => { "name" => "single_test", "build" => "Ruby Appium Sample" } } appium_driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "https://API_KEY:API_SECRET@hub.testingbot.com/wd/hub" }}, true) driver = appium_driver.start_driver driver.find_element(:name, "inputA").send_keys 5 driver.find_element(:name, "inputB").send_keys 10 driver.quit ``` ## Public Internet If you have your app available on a public website, you can specify this in the desired capabilities: ```ruby require 'appium_lib' caps = { "platformName" => "iOS", "appium:deviceName" => "iPhone 15", "appium:platformVersion" => "18.0", "appium:app" => "https://...", # specify the link to your app here "tb:options" => { "name" => "single_test", "build" => "Ruby Appium Sample" } } appium_driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "https://API_KEY:API_SECRET@hub.testingbot.com/wd/hub" }}, true) driver = appium_driver.start_driver driver.find_element(:name, "inputA").send_keys 5 driver.find_element(:name, "inputB").send_keys 10 driver.quit ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/help/tunnel # Using TestingBot Tunnel with Real Devices With TestingBot tunnel, we create a secure connection between a real device running in our datacenter, and your computer and internal network. This way, you can run tests against apps using internal (staging) APIs, from our real mobile devices. Or test websites running on your local network through Safari/Chrome running on a real device. TestingBot Tunnel in combination with Real Devices is available for both **public devices** and **private devices**. More information regarding our TestingBot Tunnel is available on our [TestingBot Tunnel documentation page](https://testingbot.com/support/tunnel). ## Known Limitations - **Localhost testing does not work on iOS**. Due to an iOS Restriction, it is currently not possible to run tests against `http(s)://localhost` with TestingBot Tunnel. A possible workaround would be to change the hosts file on the machine running TestingBot Tunnel, and add a custom hostname (for example `mylocalwebsite` and the IP address `127.0.0.1`). Then configure your tests to use `http(s)://mylocalwebsite/...` instead of localhost. ## Security and Privacy (Advisory) At TestingBot, we clean each device after every test. We take the utmost care to clean as much from the device as we can, to make sure your test runs without any leftover artifacts from previous tests. However, due to the complexity of automatically cleaning different mobile device models, we cannot 100% guarantee that all data generated from a previous session is removed from our **public devices**. Using our TestingBot tunnel in combination with a public device might have potential security risks. We recommend to [opt for private, dedicated devices](https://testingbot.com/enterprise/private-device-cloud) for maximum security. With **private devices** , your company will be the only one running tests on the device. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/help/faq # Mobile App Testing FAQ TestingBot provides mobile application testing on both **virtual devices** as well as **physical devices**. Virtual devices are Android emulators and iOS simulators. TestingBot provides a large collection of these, available for mobile app testing, both automated and manual testing. This FAQ document will list questions and answers regarding the mobile native app testing service. ## Can I run Android emulator tests using Espresso instead of Appium? TestingBot allows you to upload native mobile Android apps and use [Espresso testing](https://testingbot.com/support/app-automate/espresso) on both Android emulators and physical devices. Of course, you can do [Appium automated app testing](https://testingbot.com/support/app-automate/appium) as well on TestingBot. ## Which apps are available on the devices? TestingBot uses stock images for Android emulators, which come equipped with the PlayStore and Chrome. Similarly, we use stock iOS simulators which come with Safari by default. TestingBot does not use jailbroken or rooted physical devices. All Android devices come with factory settings, with PlayStore enabled (if available) and Chrome. The iOS devices come with Safari, the Safari version depends on the iOS version installed on the device. ## How does manual testing work? We provide access to a realtime view of an Android emulator, iOS simulator, Android physical device and iOS device. You can interact with the device, from your own mouse and keyboard, without installing any plugins or software. The screens are streamed either through WebRTC or VNC, depending on your network connectivity settings. ## When are new devices added? Our public device pool contains a collection of the most popular devices used, backed by public statistics of mobile device usage. When new devices are released to the public, we aim to procure, setup and make these available to our customers in less than 2 weeks. The criteria we use for selecting which devices to add depends on the popularity of the device and the demand for testing on such a device. If you are looking to test on a specific device, please do not hesitate to [reach out to us](https://testingbot.com/contact/new). ## Is it possible to test push notifications? It is possible to test push notifications in the following scenarios: - **Android Physical Devices** It will work automatically. Simply install the app and allow push notifications. - **iOS Physical Devices** This is only possible for private devices, which require an [Enterprise plan](https://testingbot.com/enterprise). [Reach out to us](https://testingbot.com/contact/new) for more information. - **Android Emulators** Unfortunately this is not supported yet. If you'd like to be a part of the beta program, please let us know. - **iOS Simulators** This is possible since Appium 1.21.0. ## Can I upload or download files from a device or sim/emu? Yes, you can upload files to devices using Appium's `pushFile` command and download files using the `pullFile` command. Please see our [Upload Files documentation](https://testingbot.com/support/app-automate/appium/upload-files) for more information. ## How is my app protected during mobile app testing? Our Real Device Cloud is located in a secure datacenter in Europe. We make sure that every device is fully protected, both physically and digitally. All traffic happens through SSL and each device is cleaned after usage: - All user apps are removed - Generated data, such as browser data, is wiped - SD card, accounts, and other data that was altered is removed TestingBot's [automated cleaning process](https://testingbot.com/support/app-automate/help/cleaning) makes sure that no data is available from an earlier test. If you think this did happen, please reach out to us so we can reset the device and check why this happened. ## Is audio supported, or microphone access? Audio is available for Android devices running Android 12 and higher. Older physical devices do not have the capability to stream audio, these will only stream a realtime video of the screen. ## My app needs to make requests to internal services. Can I use a VPN for this? We suggest using the [TestingBot Tunnel](https://testingbot.com/support/tunnel). It will set up the device or sim/emu to proxy all requests through the tunnel to your internal services. An alternative solution is to [whitelist our IP addresses](https://testingbot.com/support/other/configuration#ip-range), which would allow our devices to reach your protected services. ## Can I use Bluetooth on the physical devices? Yes you can, please [reach out](https://testingbot.com/contact/new) to discuss your specific use case. ## Can I trigger a disconnect or reconnect from a charging cable? This is currently not supported. ## Can I change the orientation of the device? Yes, both on simulators/emulators as well as on physical devices. You can change the orientation to either `PORTRAIT` or `LANDSCAPE`. You can do this via our manual testing feature, or [change the screen orientation during an automated test](https://testingbot.com/support/app-automate/appium/screen-orientation). ## Are code coverage reports by JaCoCo supported? We currently do not support this. ## Can I change the geolocation during my test? Yes, you can [change the geolocation](https://testingbot.com/support/web-automate/selenium/test-options#geo) either during manual testing or via automated tests. ## Why do I sometimes see a lock/PIN screen? On trial devices, in our public device pool, we need to make some areas password protected to prevent users from changing certain settings. The password protection is not in place on non-trial devices. ## Can I test payments on the physical devices? Payments can be tested on private devices only. Please [reach out to us](https://testingbot.com/contact/new) for more information. ## Can I inspect the mobile app in your manual testing feature? Yes, you can inspect the mobile app from the manual testing feature. Simply open the **DevTools** option and click the **Inspector** tab. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/help/cleaning # Real Device Cleaning Process TestingBot automatically cleans each physical mobile device after every test session. We do this to ensure each new test starts with a clean state, without any leftover data from previous sessions. The cleaning process we use is automated for both iOS and Android. For devices in the public device pool, each test session automatically ends with a thorough cleaning process where we delete any apps installed by the user, together with various other files and data that were generated. If you are using a private device, there's the possibility to alter or completely disable the cleaning process. You would for example use this when you want to make sure that a specific state is kept between various test sessions. ## Cleaning Process TestingBot uses a proprietary process to clean its devices. Below are the various processes that happen when a device is being cleaned. - All apps that have been installed by the customer are deleted. Some applications, which are used by TestingBot to operate the device, are kept. - The browser history, together with any cached website data and cookies are removed. - AppleID sessions are removed. - Network settings such as WiFi and DNS configurations are reset to their default state. - The system locale, language and time zone are reset to their default. (English, UTC+2). - Any mock GPS location is reset to Brussels, Belgium. - Media files such as Photos, Videos and other Files are removed from the device. TestingBot's cleaning process aims to clean as much as possible from the device. If you prefer the guarantee of a pristine device, please consider using a [private mobile device](https://testingbot.com/enterprise/private-device-cloud). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/help/app-resigning # iOS App Resigning When you [upload an iOS app](https://testingbot.com/support/app-automate/help/upload) to TestingBot, we will automatically re-sign the `.ipa` file with our own provisioning profile, to be able to install and run your app on our mobile devices. Because of this procedure, the [entitlements](https://developer.apple.com/documentation/bundleresources/entitlements) of your iOS app will be removed. You can choose to disable this re-signing, if your app is signed using the [Apple Developer Enterprise Program](https://developer.apple.com/programs/enterprise/). This way you can test **push notifications** or [universal links](https://developer.apple.com/ios/universal-links/) on our physical devices. ## Disable iOS Resigning [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby capabilities = { "tb:options" => { "resigningEnabled" => false } } ``` ```java Map tbOptions = new HashMap<>(); tbOptions.put("resigningEnabled", false); options.setCapability("tb:options", tbOptions); ``` ```php $capabilities = [ "tb:options" => [ "resigningEnabled" => false ] ]; ``` ```python capabilities = { "tb:options": { "resigningEnabled": False } } ``` ```javascript const capabilities = { "tb:options": { "resigningEnabled": false } }; ``` ```csharp var tbOptions = new Dictionary { ["resigningEnabled"] = false }; options.AddAdditionalAppiumOption("tb:options", tbOptions); ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/smart-tv # Smart TV Testing Run tests on physical Smart TV devices with TestingBot, using the open-source test framework Appium. Currently TestingBot supports running automated tests on physical AppleTV 4k devices. To get started with testing your Smart TV app in the cloud, you will need to follow these steps: - Upload your native Smart TV app, either to a public URL or using TestingBot Storage. - Create an Appium test script that will connect to TestingBot. - The results will be available in the TestingBot dashboard, together with a video of the test and logs. ## Upload Smart TV App To run Smart TV tests on TestingBot, you will first have to upload your `.ipa` file to TestingBot Storage. [cURL](https://testingbot.com#) ```bash $ curl -X POST "https://api.testingbot.com/v1/storage" \ -u key:secret -F "file=@/path/to/app/file/tvos.ipa" ``` This call will return a unique identifier for your SmartTV app (`{"app_url": "tb://..."}`), which you can use in the capabilities in your test (see example below). More information regarding this API call and other similar calls (update uploaded file, delete uploaded file, list uploaded files) are available in [the TestingBot API documentation](https://testingbot.com/support/api/storage#upload). Note: uploads to TestingBot Storage are automatically deleted after 60 days. ## Run your first SmartTV test Once you've [uploaded your Smart TV app](https://testingbot.com#upload), you can start with running your first test on the TestingBot SmartTV cloud. TestingBot currently only supports running tests on physical AppleTV 4k devices. Please see the example tests below, where we use the [TestingBot tvOS example app](https://github.com/testingbot/tvos-example) to input two numbers in the calculator and verify its sum. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby require 'rubygems' require 'appium_lib' caps = {} caps['name'] = 'Ruby tvOS Example' caps['deviceName'] = 'Apple TV 4K' caps['platformName'] = 'tvOS' caps['version'] = '17.4' caps['app'] = 'https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa' caps['realDevice'] = true appium_driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "https://key:secret@hub.testingbot.com/wd/hub" }}, true) driver = appium_driver.start_driver wait = Selenium::WebDriver::Wait.new(:timeout => 30) wait.until { driver.find_element(:accessibility_id, "inputField1").displayed? } inputField1 = driver.find_element(:accessibility_id, "inputField1") inputField1.click inputField1.send_keys 5 driver.execute_script('mobile: pressButton', { name: 'down' }) driver.execute_script('mobile: pressButton', { name: 'select' }) driver.switch_to.active_element.send_keys 10 driver.execute_script('mobile: pressButton', { name: 'down' }) driver.execute_script('mobile: pressButton', { name: 'select' }) result = driver.find_element(:accessibility_id, "resultField") if (!result.nil?) && (result.text.to_i == 15) puts "Test Passed" else puts "Test Failed" end driver.quit ``` ```javascript const wdio = require('webdriverio'); const caps = { name: 'Node.js tvOS Example', deviceName: 'Apple TV 4K', platformName: 'tvOS', platformVersion: '17.4', app: 'https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa', realDevice: true }; const options = { protocol: 'https', hostname: 'hub.testingbot.com', port: 443, path: '/wd/hub', user: 'key', // Replace 'key' with your TestingBot key key: 'secret', // Replace 'secret' with your TestingBot secret capabilities: caps }; (async () => { // Start the Appium driver const driver = await wdio.remote(options); try { // Wait for the first input field to be displayed const inputField1 = await driver.$('~inputField1'); await inputField1.waitForDisplayed({ timeout: 30000 }); // Interact with the first input field await inputField1.click(); await inputField1.setValue(5); // Navigate down and select await driver.execute('mobile: pressButton', { name: 'down' }); await driver.execute('mobile: pressButton', { name: 'select' }); // Send keys to the active element await driver.switchToParentFrame(); // Ensure focus is on the active frame await driver.activeElement().setValue(10); // Navigate down and select the result await driver.execute('mobile: pressButton', { name: 'down' }); await driver.execute('mobile: pressButton', { name: 'select' }); // Verify the result const result = await driver.$('~resultField'); const resultText = await result.getText(); if (resultText && parseInt(resultText, 10) === 15) { console.log('Test Passed'); } else { console.log('Test Failed'); } } catch (err) { console.error('Error occurred:', err); } finally { // Quit the driver await driver.deleteSession(); } })(); ``` ```python from appium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC # Define capabilities caps = { "name": "Python tvOS Example", "deviceName": "Apple TV 4K", "platformName": "tvOS", "platformVersion": "17.4", "app": "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa", "realDevice": True } # Appium server URL server_url = "https://key:secret@hub.testingbot.com/wd/hub" # Initialize the driver driver = webdriver.Remote(server_url, caps) try: # Wait for the first input field to be displayed wait = WebDriverWait(driver, 30) input_field1 = wait.until(EC.presence_of_element_located((By.ACCESSIBILITY_ID, "inputField1"))) # Interact with the first input field input_field1.click() input_field1.send_keys("5") # Navigate down and select the second input field driver.execute_script('mobile: pressButton', {'name': 'down'}) driver.execute_script('mobile: pressButton', {'name': 'select'}) # Send keys to the active element (second input field) driver.switch_to.active_element.send_keys("10") # Navigate down and select the result field driver.execute_script('mobile: pressButton', {'name': 'down'}) driver.execute_script('mobile: pressButton', {'name': 'select'}) # Verify the result result_field = driver.find_element(By.ACCESSIBILITY_ID, "resultField") result_text = result_field.text if result_text and int(result_text) == 15: print("Test Passed") else: print("Test Failed") except Exception as e: print(f"An error occurred: {e}") finally: # Quit the driver driver.quit() ``` ```java import io.appium.java_client.AppiumDriver; import io.appium.java_client.MobileBy; import io.appium.java_client.remote.MobileCapabilityType; import org.openqa.selenium.By; import org.openqa.selenium.WebElement; import org.openqa.selenium.remote.DesiredCapabilities; import java.net.URL; import java.time.Duration; import org.openqa.selenium.support.ui.ExpectedConditions; import org.openqa.selenium.support.ui.WebDriverWait; public class TvOSTest { public static void main(String[] args) { AppiumDriver driver = null; try { // Set capabilities DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("name", "Java tvOS Example"); caps.setCapability("deviceName", "Apple TV 4K"); caps.setCapability("platformName", "tvOS"); caps.setCapability("platformVersion", "17.4"); caps.setCapability("app", "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa"); caps.setCapability("realDevice", true); URL serverUrl = new URL("https://key:secret@hub.testingbot.com/wd/hub"); driver = new AppiumDriver<>(serverUrl, caps); // Wait for the first input field to be displayed WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(30)); WebElement inputField1 = wait.until(ExpectedConditions.presenceOfElementLocated(MobileBy.AccessibilityId("inputField1"))); inputField1.click(); inputField1.sendKeys("5"); // Navigate down and select the second input field driver.executeScript("mobile: pressButton", Map.of("name", "down")); driver.executeScript("mobile: pressButton", Map.of("name", "select")); // Send keys to the active element (second input field) WebElement activeElement = driver.switchTo().activeElement(); activeElement.sendKeys("10"); // Navigate down and select the result field driver.executeScript("mobile: pressButton", Map.of("name", "down")); driver.executeScript("mobile: pressButton", Map.of("name", "select")); WebElement resultField = driver.findElement(MobileBy.AccessibilityId("resultField")); String resultText = resultField.getText(); if (resultText != null && Integer.parseInt(resultText) == 15) { System.out.println("Test Passed"); } else { System.out.println("Test Failed"); } } catch (Exception e) { e.printStackTrace(); } finally { // Quit the driver if (driver != null) { driver.quit(); } } } } ``` ```csharp using OpenQA.Selenium; using OpenQA.Selenium.Appium; using OpenQA.Selenium.Appium.Enums; using OpenQA.Selenium.Appium.iOS; using OpenQA.Selenium.Support.UI; using System; using System.Collections.Generic; class TvOSTest { static void Main(string[] args) { AppiumDriver driver = null; try { // Set capabilities var caps = new AppiumOptions(); caps.AddAdditionalCapability(MobileCapabilityType.PlatformName, "tvOS"); caps.AddAdditionalCapability(MobileCapabilityType.DeviceName, "Apple TV 4K"); caps.AddAdditionalCapability(MobileCapabilityType.PlatformVersion, "17.4"); caps.AddAdditionalCapability(MobileCapabilityType.App, "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa"); caps.AddAdditionalCapability("realDevice", true); caps.AddAdditionalCapability("name", "C# tvOS Example"); // Initialize driver var serverUri = new Uri("https://key:secret@hub.testingbot.com/wd/hub"); driver = new IOSDriver(serverUri, caps); // Wait for the first input field to be displayed var wait = new WebDriverWait(driver, TimeSpan.FromSeconds(30)); var inputField1 = wait.Until(d => d.FindElement(MobileBy.AccessibilityId("inputField1"))); // Interact with the first input field inputField1.Click(); inputField1.SendKeys("5"); // Navigate down and select the second input field driver.ExecuteScript("mobile: pressButton", new Dictionary { { "name", "down" } }); driver.ExecuteScript("mobile: pressButton", new Dictionary { { "name", "select" } }); // Send keys to the active element (second input field) var activeElement = driver.SwitchTo().ActiveElement(); activeElement.SendKeys("10"); // Navigate down and select the result field driver.ExecuteScript("mobile: pressButton", new Dictionary { { "name", "down" } }); driver.ExecuteScript("mobile: pressButton", new Dictionary { { "name", "select" } }); // Verify the result var resultField = driver.FindElement(MobileBy.AccessibilityId("resultField")); var resultText = resultField.Text; if (!string.IsNullOrEmpty(resultText) && int.Parse(resultText) == 15) { Console.WriteLine("Test Passed"); } else { Console.WriteLine("Test Failed"); } } catch (Exception ex) { Console.WriteLine($"An error occurred: {ex.Message}"); } finally { // Quit the driver driver?.Quit(); } } } ``` ## View SmartTV Test Results Once your test has finished running, you will see a new test entry in the [TestingBot dashboard.](https://testingbot.com/members). If you click the test, you will see detailed information about the test, including a video recording of the test, logs and screenshots. You can also use the [TestingBot API](https://testingbot.com/support/api) to programmatically fetch the test results. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/smart-tv/options # Smart TV Test Options Please see the examples below on how to customize your test run on the available Smart TV devices. ## Specifying Appium Version TestingBot will use the most recent, compatible, Appium version according to the device, OS and version you specify. If you'd like to specify your own Appium version, you can do this with the `appiumVersion` capability. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "tvOS", deviceName: "Apple TV 4K", browserVersion: "17.4", app: 'https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa', "tb:options" => { "appiumVersion" : "2.0" } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("appiumVersion", "2.0"); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "tvOS"); caps.setCapability("tb:options", tbOptions); caps.setCapability("deviceName", "Apple TV 4K"); caps.setCapability("browserVersion", "17.4"); caps.setCapability("app", "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa"); ``` ```python tbOptions = { 'name': 'W3C Sample', 'appiumVersion': "2.0" } chromeOpts = { 'app': "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa", 'platformName': "tvOS", 'browserVersion': "17.4", 'deviceName': "Apple TV 4K", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "app": 'https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa', "platformName": 'tvOS', "deviceName": 'Apple TV 4K', "version": '17.4', /** Google requires "w3c" to be set in "goog:chromeOptions" as true if you're using ChromeDriver version 74 or lower. * Based on this commit: https://chromium.googlesource.com/chromium/src/+/2b49880e2481658e0702fd6fe494859bca52b39c * ChromeDriver now uses w3c by default from version 75+ so setting this option will no longer be a requirement **/ "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "appiumVersion": "2.0" } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "17.4", PlatformName = "tvOS", DeviceName = "Apple TV 4K", App = "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["appiumVersion"] = "2.0" }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` ### Appium 2 Versions We currently support these Appium 2 versions: - 2.11.3 - 2.10.3 - 2.9.0 - 2.5.1 - 2.4.1 - 2.3.0 - 2.2.1 If you specify `"appiumVersion": "latest"`, TestingBot will automatically use the latest Appium version. You can also use `latest-1`, `latest-2`, ... to test on the next most recent versions of Appium. ### Geolocation Testing TestingBot provides an option where you can specify from which country you'd like to run the test from. We will configure the Smart TV device to use a proxy that will route all traffic through the specified country. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "tvOS", app: "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa", deviceName: "Apple TV 4K", version: "17.4", "tb:options" => { "testingbot.geoCountryCode" : "DE" } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("testingbot.geoCountryCode", "DE"); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "tvOS"); caps.setCapability("tb:options", tbOptions); caps.setCapability("app", "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa"); ``` ```python tbOptions = { 'testingbot.geoCountryCode': "DE" } chromeOpts = { 'app': "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa", 'platformName': "tvOS", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "app": 'https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa', "platformName": 'tvOS', "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "testingbot.geoCountryCode": "DE" } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { Version = "17.4", PlatformName = "tvOS", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["testingbot.geoCountryCode"] = "DE" }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` Specify `testingbot.geoCountryCode` with one of the following country codes: - **'\*'** : this will take a random country from the list below - **'AU'** : Australia - **'BE'** : Belgium - **'BR'** : Brazil - **'CA'** : Canada - **'CL'** : Chile - **'FR'** : France - **'DE'** : Germany - **'IN'** : India - **'IT'** : Italy - **'JP'** : Japan - **'NO'** : Norway - **'SG'** : Singapore - **'ZA'** : South Africa - **'SE'** : Sweden - **'CH'** : Switzerland - **'AE'** : United Arab Emirates - **'GB'** : United Kingdom - **'US'** : United States **Important:** this does not work on tvOS 4.4 ### Change Test Name Add a name to this test, which will show up in the TestingBot member area and API. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "tvOS", app: "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa", version: "17.4", deviceName: "Apple TV 4K", "tb:options" => { "name" : "My Test Name" } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("name", "My Test Name"); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "tvOS"); caps.setCapability("tb:options", tbOptions); caps.setCapability("app", "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('tvOS'); $capabilities->setCapability('tb:options', array( 'name' => "My Test Name" )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'name': "My Test Name" } chromeOpts = { 'app': "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa", 'platformName': "tvOS", 'deviceName': "Apple TV 4K", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "app": 'https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa', "platformName": 'tvOS', "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "name": "My Test Name" } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { Version = "17.4", PlatformName = "tvOS", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["name"] = "My Test Name" }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` Value Type | Default Value || string | unnamed test | ### Group Tests A key you can use to group certain tests in the same build (for example in Jenkins). The builds will appear in our member area. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "tvOS", app: "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa", version: "17.4", deviceName: "Apple TV 4K", "tb:options" => { "build" : "My First Build" } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("build", "My First Build"); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "tvOS"); caps.setCapability("tb:options", tbOptions); caps.setCapability("app", "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('tvOS'); $capabilities->setCapability('tb:options', array( 'build' => "My First Build" )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'build': "My First Build" } chromeOpts = { 'app': "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa", 'platformName': "tvOS", 'deviceName': "Apple TV 4K", 'version': '17.4', 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "app": 'https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa', "platformName": 'tvOS', "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "build": "My First Build" } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { Version = "17.4", PlatformName = "tvOS", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["build"] = "My First Build" }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` Value Type || string | ### Idle Timeout The maximum amount of time a device will wait before proceeding to the next step in your test. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "tvOS", app: "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa", version: "17.4", deviceName: "Apple TV 4K", "tb:options" => { "idletimeout" : 130 } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("idletimeout", 130); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "tvOS"); caps.setCapability("tb:options", tbOptions); caps.setCapability("app", "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('tvOS'); $capabilities->setCapability('tb:options', array( 'idletimeout' => 130 )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'idletimeout': 130 } chromeOpts = { 'app': "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa", 'platformName': "tvOS", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "app": 'https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa', "platformName": 'tvOS', "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "idletimeout": 130 } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { Version = "17.4", PlatformName = "tvOS", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["idletimeout"] = 130 }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` Value Type | Default Value || int (specify number of seconds) | 130 seconds | ### Maximum Test Duration The maximum duration for a single test. This is a safeguard to prevent bad tests from using up your credits. We generally recommend to keep tests short (less than 10 minutes). It's better to split up large tests in smaller individual tests. This keeps your tests fast and allows for more parallelization of your tests. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "tvOS", app: "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa", version: "17.4", deviceName: "Apple TV 4K", "tb:options" => { "maxduration" : 1800 } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("maxduration", 1800); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "tvOS"); caps.setCapability("tb:options", tbOptions); caps.setCapability("app", "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('tvOS'); $capabilities->setCapability('tb:options', array( 'maxduration' => 1800 )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'maxduration': 1800 } chromeOpts = { 'app': "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa", 'platformName': "tvOS", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "app": 'https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa', "platformName": 'tvOS', "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "maxduration": 1800 } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { Version = "17.4", PlatformName = "tvOS", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["maxduration"] = 1800 }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` Value Type | Default Value || int (specify number of seconds) | 1800 seconds (30 minutes) | ### Custom Metadata Send along custom data, for example your release, server, commit hash, ... This will show up on the test detail page in the TestingBot member area. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "tvOS", app: "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa", version: "17.4", deviceName: "Apple TV 4K", "tb:options" => { "extra" : "Extra Information" } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("extra", "Extra Information"); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "tvOS"); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "chrome"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('tvOS'); $capabilities->setCapability('tb:options', array( 'extra' => "Extra Information" )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'extra': "Extra Information" } chromeOpts = { 'app': "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa", 'platformName': "tvOS", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "app": 'https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa', "platformName": 'tvOS', "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "extra": "Extra Information" } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { Version = "17.4", PlatformName = "tvOS", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["extra"] = "Extra Information" }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` Value Type || string | ### Taking screenshots during your tests By default, TestingBot does not capture screenshots at every step of your mobile test. If you wish to take a screenshot for every step, please add this capability (set to `true`) to your request. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "tvOS", app: "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa", version: "17.4", deviceName: "Apple TV 4K", "tb:options" => { screenshot: true } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("screenshot", true); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "tvOS"); caps.setCapability("tb:options", tbOptions); caps.setCapability("app", "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('tvOS'); $capabilities->setCapability('tb:options', array( 'screenshot' => true )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'name': 'W3C Sample', 'screenshot': True } chromeOpts = { 'app': "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa, 'platformName': "tvOS", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "app": 'https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa', "platformName": 'tvOS', "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "screenshot": true } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { Version = "23", PlatformName = "tvOS", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["screenshot"] = true }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` Value Type | Default Value || boolean | false | ### Make a video of your tests By default we record a video of your test, which is accessible in the member area. If you do not wish to have this, you can disable it with this option. Video should not slow down your test considerably. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "tvOS", app: "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa", version: "17.4", "tb:options" => { screenrecorder: true } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("screenrecorder", true); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "tvOS"); caps.setCapability("tb:options", tbOptions); caps.setCapability("app", "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('tvOS'); $capabilities->setCapability('tb:options', array( 'screenrecorder' => true )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'name': 'W3C Sample', 'screenrecorder': True } chromeOpts = { 'app': "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa", 'platformName': "tvOS", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "app": 'https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa', "platformName": 'tvOS', "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "screenrecorder": true } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { Version = "17.4", PlatformName = "tvOS", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["screenrecorder"] = true }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` Value Type | Default Value || boolean | true | ### Test Privacy Make the test results for this test public so that everyone can access the results. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "tvOS", app: "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa", version: "17.4", "tb:options" => { "public" : false } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("public", false); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "tvOS"); caps.setCapability("tb:options", tbOptions); caps.setCapability("app", "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('tvOS'); $capabilities->setCapability('tb:options', array( 'public' => 130 )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'public': False } chromeOpts = { 'app': "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa", 'platformName': "tvOS", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "app": 'https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa', "platformName": 'tvOS', "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "public": false } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { Version = "17.4", PlatformName = "tvOS", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["public"] = false }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` Value Type | Default Value || boolean | false | ### Customize Logging By default, TestingBot records logs of all test actions and its drivers. Set this option to `false` if you don't want TestingBot to record anything (for example, if you have sensitive data). You will not see any test logs in our member dashboard. Set to `strip-parameters` to prevent the POST/GET parameters from being logged on the TestingBot test detail page (does not affect other logs like Appium logs, Chromedriver logs, ...). [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "tvOS", app: "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa", version: "17.4", deviceName: "Apple TV 4K", "tb:options" => { "recordLogs" : true } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("recordLogs", true); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "tvOS"); caps.setCapability("tb:options", tbOptions); caps.setCapability("app", "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('tvOS'); $capabilities->setCapability('tb:options', array( 'recordLogs' => true )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'recordLogs': True } chromeOpts = { 'app': "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa", 'platformName': "tvOS", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "app": 'https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa', "platformName": 'tvOS', "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "recordLogs": true } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { Version = "17.4", PlatformName = "tvOS", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["recordLogs"] = true }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` Value Type | Default Value | Possible Values: || string | "true" | `true`, `false`, or `strip-parameters` | ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/smart-tv/remote-control-navigation # TvOS (Apple TV 4K) During an Appium test you can use the buttons actions to simulate remote control navigation on the Apple TV. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby driver.execute_script('mobile: pressButton', { name: 'down' }) driver.execute_script('mobile: pressButton', { name: 'select' }) ``` ```javascript await driver.execute('mobile: pressButton', { name: 'down' }); await driver.execute('mobile: pressButton', { name: 'select' }); ``` ```python driver.execute_script('mobile: pressButton', {'name': 'down'}) driver.execute_script('mobile: pressButton', {'name': 'select'}) ``` ```java driver.executeScript("mobile: pressButton", Map.of("name", "down")); driver.executeScript("mobile: pressButton", Map.of("name", "select")); ``` ```csharp driver.ExecuteScript("mobile: pressButton", new Dictionary { { "name", "down" } }); driver.ExecuteScript("mobile: pressButton", new Dictionary { { "name", "select" } }); ``` The following buttons are available: | Button Name | Behaviour | | --- | --- | | Home | Return to the home screen. | | Menu | Return to the previous operation or screen. In the context of tvOS, it works as a traditional back button. | | Select | Select the item with the current focus. | | Up | Move the focus upward in the UI. | | Down | Move the focus downward in the UI. | | Left | Move the focus left in the UI. | | Right | Move the focus right in the UI. | | Play/Pause | Control media playback. Play/Pause is a toggle. | ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/appium Documentation # Appium Cloud Testing Appium is the open-source W3C WebDriver framework for native, hybrid and mobile-web applications. TestingBot runs Appium tests on real iPhones, iPads and Android devices in our EU datacenter, alongside iOS simulators and Android emulators for fast iteration. Bring your existing Appium tests in Java, Python, Node.js, Ruby, C#, PHP or Robot Framework and point them at the TestingBot grid. ## Quick start: run your first Appium test Four steps to point an existing Appium test at the TestingBot cloud: 1. **Sign up and grab your API key.** [Create a free account](https://testingbot.com/users/sign_up). Your API key and secret are in the dashboard under Account, then Keys and Authentication. 2. **Upload your app binary to TestingBot Storage.** Upload your `.ipa` (iOS) or `.apk` (Android) to [TestingBot Storage](https://testingbot.com/support/api/storage#upload). The response returns a `tb://` URL you reference as the `appium:app` capability. 3. **Point Appium at the TestingBot endpoint.** Connect your Appium client to `https://hub.testingbot.com/wd/hub`. Set `platformName`, `appium:deviceName`, `appium:platformVersion` and `appium:app` (the `tb://` URL). Pass your key and secret via `tb:options`. 4. **Run the test and view results.** TestingBot streams a live video, screenshots, device logs and Appium command logs into the dashboard while the test runs and after it finishes. ## Pick a language or framework Each card below links to a complete setup guide for that language or framework, including a working example, capability reference and CI integration. ### [Java](https://testingbot.com/support/app-automate/appium/java) Appium with the Java Appium client. JUnit, TestNG and Cucumber friendly. - [Java tutorial](https://testingbot.com/support/app-automate/appium/java) - [TestNG](https://testingbot.com/support/app-automate/appium/java/testng) ### [Node.js](https://testingbot.com/support/app-automate/appium/nodejs) Appium with the Node.js Appium or WebdriverIO clients. - [Node.js tutorial](https://testingbot.com/support/app-automate/appium/nodejs) - [CodeceptJS](https://testingbot.com/support/app-automate/appium/nodejs/codeceptjs) - [WebdriverIO](https://testingbot.com/support/app-automate/appium/nodejs/webdriverio) ### [C# / .NET](https://testingbot.com/support/app-automate/appium/csharp) Appium with the C# Appium client. NUnit, Reqnroll and SpecFlow friendly. - [C# tutorial](https://testingbot.com/support/app-automate/appium/csharp) - [NUnit](https://testingbot.com/support/app-automate/appium/csharp/nunit) - [Reqnroll](https://testingbot.com/support/app-automate/appium/csharp/reqnroll) - [SpecFlow](https://testingbot.com/support/app-automate/appium/csharp/specflow) ### [Python](https://testingbot.com/support/app-automate/appium/python) Appium with the Python Appium client and pytest. - [Python tutorial](https://testingbot.com/support/app-automate/appium/python) - [Robot Framework](https://testingbot.com/support/app-automate/appium/robotframework) ### [Ruby](https://testingbot.com/support/app-automate/appium/ruby) Appium with the Ruby `appium_lib` gem. - [Ruby tutorial](https://testingbot.com/support/app-automate/appium/ruby) ### [PHP](https://testingbot.com/support/app-automate/appium/php) Appium with the PHP WebDriver client. - [PHP tutorial](https://testingbot.com/support/app-automate/appium/php) ![Robot Framework](https://testingbot.com/assets/wizard/robotframework-08821f4942343bcb1f828470b5cbff3163369c14884b7ca285be190017ee7a94.png) ### [Robot Framework](https://testingbot.com/support/app-automate/appium/robotframework) Keyword-driven testing with AppiumLibrary on top of Robot Framework. - [Robot Framework tutorial](https://testingbot.com/support/app-automate/appium/robotframework) ## Preparing your app Before uploading, make sure your `.ipa` or `.apk` is built for testing (not release-signed for the App Store) so it can be installed on TestingBot devices. See [preparing a mobile app for testing](https://testingbot.com/support/app-automate/help/prepare) for the iOS provisioning profile and Android signing requirements. ## Useful Appium capabilities Appium exposes [a comprehensive list of capabilities](https://appium.io/docs/en/2.0/guides/caps/) for configuring a test. The capabilities below are the ones we see come up most often: ### Android-specific - `appium:appActivity` and `appium:appPackage`: by default Appium extracts the launch activity from your APK. Set these explicitly if extraction fails or if you want to start in a non-default activity. - `appium:chromeOptions`: extra ChromeDriver options when testing the in-app Chromium web view. - `appium:otherApps`: a JSON array of additional app URLs to install alongside the app under test (useful for testing share-targets or deep links to other apps). ### Android and iOS - `appium:locale`: the language of the simulator or device (for example `fr_CA`). This sets the locale on the entire device or simulator, except on real iOS devices. For real iOS devices, TestingBot passes `-AppleLocale` as a launch argument to the app instead. - `appium:newCommandTimeout`: how many seconds Appium waits for a new command from the client before assuming the client quit and ending the session. - `appium:autoGrantPermissions` (Android) or `appium:autoAcceptAlerts` (iOS): silently grant or accept permission popups so they do not block the test. See [permission popups](https://testingbot.com/support/app-automate/appium/permission-popups). - `realDevice: true` inside `tb:options`: route the test to a physical iPhone, iPad or Android device instead of a simulator or emulator. ## Frequently asked questions What is Appium and what is it used for? Appium is an open-source cross-platform test framework for native, hybrid and mobile-web applications. It implements the W3C WebDriver protocol so you can use the same client library to drive iOS and Android apps from Java, Python, Node.js, Ruby, C#, PHP and other languages. TestingBot runs Appium on a cloud grid of real devices and simulators so you do not have to maintain local Appium servers. Which languages and frameworks does Appium support on TestingBot? All Appium client libraries work against the TestingBot grid: [Java](https://testingbot.com/support/app-automate/appium/java), [Python](https://testingbot.com/support/app-automate/appium/python), [Node.js](https://testingbot.com/support/app-automate/appium/nodejs) (including [WebdriverIO](https://testingbot.com/support/app-automate/appium/nodejs/webdriverio) and [CodeceptJS](https://testingbot.com/support/app-automate/appium/nodejs/codeceptjs)), [Ruby](https://testingbot.com/support/app-automate/appium/ruby), [C#](https://testingbot.com/support/app-automate/appium/csharp) (including [NUnit](https://testingbot.com/support/app-automate/appium/csharp/nunit), [Reqnroll](https://testingbot.com/support/app-automate/appium/csharp/reqnroll) and [SpecFlow](https://testingbot.com/support/app-automate/appium/csharp/specflow)), [PHP](https://testingbot.com/support/app-automate/appium/php) and [Robot Framework](https://testingbot.com/support/app-automate/appium/robotframework). Can I run Appium tests on real iPhones and real Android devices? Yes. TestingBot runs Appium tests on physical iPhones, iPads and Android phones and tablets housed in our EU datacenter, alongside iOS simulators and Android emulators. Use `realDevice: true` in your capabilities to target real hardware. See [real device testing](https://testingbot.com/real-device-testing) for the full real-hardware overview. How do I upload my app binary for Appium tests? Upload your `.ipa` or `.apk` to [TestingBot Storage](https://testingbot.com/support/api/storage#upload) with your API key and secret. The response returns a `tb://` URL that you pass as the `appium:app` capability in your test. See [uploading apps for Appium](https://testingbot.com/support/app-automate/appium/upload-files) for details. Which Appium version does TestingBot run? TestingBot automatically selects the latest Appium 2 release compatible with the device, OS and OS version you request. Pin a specific version with the [`appiumVersion` capability](https://testingbot.com/support/app-automate/appium/appium-versions) if you need an exact release. The full list of supported versions is on the [Appium versions page](https://testingbot.com/support/app-automate/appium/appium-versions). Can I test the mobile web (Safari, Chrome) with Appium on TestingBot? Yes. Set `browserName` to `safari` (iOS) or `chrome` (Android) and omit the `app` capability. See [mobile browser testing](https://testingbot.com/support/web-automate/mobile) for the full setup. How do I handle permission popups, push notifications or app upgrades in Appium tests? TestingBot supports the standard Appium capabilities for these flows: [auto-grant or auto-accept permission popups](https://testingbot.com/support/app-automate/appium/permission-popups), [trigger and interact with push notifications](https://testingbot.com/support/app-automate/appium/push-notifications), and [test app upgrades](https://testingbot.com/support/app-automate/appium/test-app-upgrades) by chaining installs of two binaries. Can I run Appium tests behind a firewall or against a staging API? Yes. Start the [TestingBot Tunnel](https://testingbot.com/support/tunnel) and pass `tunnelIdentifier` in your `tb:options` capabilities. The cloud device will route HTTP traffic through your tunnel so it can reach internal staging APIs and local development servers. ## Next steps - [ **Capabilities builder** Generate ready-to-paste Appium capabilities](https://testingbot.com/support/app-automate/appium/capabilities) - [ **tb:options reference** All TestingBot-specific Appium options](https://testingbot.com/support/app-automate/appium/options) - [ **Appium versions** Pin a specific Appium release](https://testingbot.com/support/app-automate/appium/appium-versions) - [ **Uploading apps** .ipa and .apk uploads to TestingBot Storage](https://testingbot.com/support/app-automate/appium/upload-files) - [ **Permission popups** Auto-grant Android and iOS permission dialogs](https://testingbot.com/support/app-automate/appium/permission-popups) - [ **Push notifications** Test FCM and APNs flows in your app](https://testingbot.com/support/app-automate/appium/push-notifications) - [ **Supported devices** Real iPhones, iPads and Android devices](https://testingbot.com/support/app-automate/devices) - [ **TestingBot Tunnel** Test against internal or staging APIs](https://testingbot.com/support/tunnel) ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/appium/capabilities # Appium Capabilities Generator Generate Appium capabilities for your mobile automated tests. Choose your device, OS version and configuration options to get the exact code you need for Java, Python, NodeJS, C#, Ruby or PHP. [![Selenium capabilities](https://testingbot.com/assets/support/selenium-266ecc0599068fc6eca0bcd9174fb46053c95cebe1c66f7a50c480f3f1a52929.svg) Selenium ](https://testingbot.com/support/web-automate/selenium/capabilities) [![Appium capabilities](https://testingbot.com/assets/support/appium-4b35a8bb56bae236bd84c539cee9b8bbb489a55ddf48e32d059c86685d230354.svg) Appium ](https://testingbot.com/support/app-automate/appium/capabilities) [![Puppeteer capabilities](https://testingbot.com/assets/support/puppeteer-0a8cdd38af69dabd46638afbb3bffc8a372902949aa599663ec379f3e875263c.svg) Puppeteer ](https://testingbot.com/support/web-automate/puppeteer/capabilities) [![Playwright capabilities](https://testingbot.com/assets/support/playwright-2dcb4e75b55ac0d1135cfa47be6137b13f46a2047ce78b96a34f51b0e2cc0d96.svg) Playwright ](https://testingbot.com/support/web-automate/playwright/capabilities) Mobile Environment ![OS selected](https://testingbot.com/assets/environments/svg/ios-383468addf160fa18d0e431f529420739d7f7d1206175f682fead627d2e99a52.svg) iOS 18.6 › iPhone 16 Loading environments... Please wait while we load the available devices and platforms. App URL or ID Appium Version 3.2.2 3.0.1 2.17.0 2.11.3 2.10.3 2.9.0 2.5.1 2.4.1 1.22.3 1.22.0 ## Project Capabilities Test Name Allows you to customize the name of the test, as it will appear in the dashboard. Build Name Group tests under the same build identifier for easy retrieval. ## Debugging Record Video Yes No Take Screenshot After Each Step Yes No Collect Logs Yes No ## Customization Locale Geolocation Testing Run the test from a different geographical location. None Random Country Australia Belgium Brazil Canada Chile France Germany India Italy Japan Norway Singapore South Africa Sweden Switzerland United Arab Emirates United Kingdom United States Screen Orientation Portrait Landscape Looking for more options? See [all Appium capabilities](https://testingbot.com/support/app-automate/appium/options). W3C Protocol W3C is the newest WebDriver protocol, TestingBot recommends using W3C. JSONWP JSONWP is the legacy protocol which is no longer actively maintained. Java Python NodeJS C# Ruby PHP ```java MutableCapabilities capabilities = new MutableCapabilities(); capabilities.setCapability(CapabilityType.PLATFORM_NAME, "iOS"); capabilities.setCapability("appium:platformVersion", "18.6"); capabilities.setCapability("appium:deviceName", "iPhone 16"); capabilities.setCapability("appium:automationName", "XCUITest"); capabilities.setCapability("appium:app", "tb://YOUR_APP_ID"); ``` --- URL: https://testingbot.com/support/app-automate/appium/appium-versions # Supported Appium Versions on TestingBot This page lists the Appium releases currently available on the TestingBot grid, the drivers installed with each, and the differences between Appium 1, 2 and 3. To pin a specific version in your tests, set the `appiumVersion` capability inside `tb:options`. ## Appium 3 TestingBot supports Appium 3. See [supported versions](https://testingbot.com#versions) below to pin Appium 3 for your tests. Appium 3 introduces a couple of breaking changes compared to Appium 2. The main changes are removal of legacy code and protocols, which were already marked as deprecated in [Appium 2](https://testingbot.com#appium2). Breaking changes in Appium 3: - Node.js and NPM minimum version bump (to 20.19 and 10) which is handled by TestingBot. - Removal of certain deprecated endpoints. - Security feature flag prefix requirements (for example replacing `adb_shell` with `uiautomator2:adb_shell`). See the [Appium 3 migration guide](https://appium.io/docs/en/latest/guides/migrating-2-to-3/) for the full list. ## Appium 2 Appium 2 is the new version of Appium, the mobile automation framework to run tests against websites and mobile apps on phones and tablets (both simulators/emulators and physical devices). The main difference between Appium 2 and Appium 1 is that Appium 2 uses the concept of plugins and drivers. This makes for a better ecosystem, with cleaner code and improved flexibility and performance. The drivers will add support for any given platform, for example an xcuitest driver will provide support for iOS XCUITest automation, an espresso driver for Android Espresso. Plugins allow for adding and customizing existing behavior of Appium. Drivers and plugins can be easily shared, installed or removed from Appium 2. Appium 2 supports iOS 14 and above and Android 6 and above. ## Appium Versions TestingBot currently supports these Appium versions: | Appium version | Status | Installed drivers | | --- | --- | --- | | 3.2.2 or `latest` | active | These drivers are installed: - `appium-xcuitest-driver`: 10.33.0 - `appium-safari-driver`: 4.1.9 - `appium-uiautomator2-driver`: 7.0.0 - `appium-flutter-driver`: 3.5.0 | | 3.0.1 or `latest-1` | active | These drivers are installed: - `appium-xcuitest-driver`: 10.0.2 - `appium-safari-driver`: 4.0.1 - `appium-uiautomator2-driver`: 5.0.0 - `appium-flutter-driver`: 2.14.2 | | 2.17.0 | active | These drivers are installed: - `appium-xcuitest-driver`: 8.4.3 - `appium-safari-driver`: 3.5.23 - `appium-uiautomator2-driver`: 4.1.2 - `appium-flutter-driver`: 2.14.2 - `appium-flutter-integration-driver`: 1.1.3 | | 2.11.3 | active | These drivers are installed: - `appium-xcuitest-driver`: 7.24.16 - `appium-safari-driver`: 3.5.17 - `appium-uiautomator2-driver`: 3.7.7 - `appium-flutter-driver`: 2.9.2 - `appium-flutter-integration-driver`: 1.1.3 | | 2.10.3 | December 1, 2025 | These drivers are installed: - `appium-xcuitest-driver`: 7.17.5 - `appium-uiautomator2-driver`: 3.5.4 - `appium-flutter-driver`: 2.6.0 | | 2.9.0 | November 1, 2025 | These drivers are installed: - `appium-xcuitest-driver`: 7.17.5 - `appium-uiautomator2-driver`: 3.5.4 - `appium-flutter-driver`: 2.6.0 | | 2.5.1 | October 1, 2025 | These drivers are installed: - `appium-xcuitest-driver`: 7.9.1 - `appium-uiautomator2-driver`: 3.0.4 - `appium-flutter-driver`: 2.5.1 | | 2.4.1 | September 1, 2025 | These drivers are installed: - `appium-xcuitest-driver`: 5.15.1 - `appium-uiautomator2-driver`: 2.43.4 - `appium-flutter-driver`: 2.4.1 | If you specify `"appiumVersion": "latest"`, TestingBot will automatically use the latest Appium version. You can also use `latest-1`, `latest-2`, ... to test on the next most recent versions of Appium. ### Appium 1 (legacy) TestingBot keeps Appium 1 available as a legacy option for older test suites, currently at version `1.22.3`. We recommend migrating new tests to Appium 3. ## Appium 3 example on TestingBot Appium 1, 2 and 3 are supported on TestingBot, on all our physical devices and virtual devices: iOS simulators and Android emulators. These Appium drivers are installed on TestingBot: - [appium-uiautomator2-driver](https://github.com/appium/appium-uiautomator2-driver) - [appium-xcuitest-driver](https://github.com/appium/appium-xcuitest-driver) - [appium-geckodriver](https://github.com/appium/appium-geckodriver) - [appium-safari-driver](https://github.com/appium/appium-safari-driver) - [Flutter](https://testingbot.com/support/app-automate/appium/flutter) These Appium plugins are pre-installed and can be activated by specifying a `tb:appiumPlugins` capability (by default empty): ```javascript { 'tb:appiumPlugins': ['images', 'execute-driver', 'universal-xml'] } ``` - [Images Plugin](https://github.com/appium/appium/tree/master/packages/images-plugin) - [Execute Driver Plugin](https://github.com/appium/appium/tree/master/packages/execute-driver-plugin) - [Universal XML Plugin](https://github.com/appium/appium/tree/master/packages/universal-xml-plugin) TestingBot defaults to the latest stable Appium release compatible with the device and OS version you request. Override it by [specifying the `appiumVersion` capability](https://testingbot.com/support/app-automate/appium/options#appiumVersion) inside `tb:options`. The example below connects to the TestingBot device grid and pins Appium 3 (3.2.2) for a mobile automation test: [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) [Ruby](https://testingbot.com#) [C#](https://testingbot.com#) ```java XCUITestOptions options = new XCUITestOptions(); options.setPlatformName("iOS"); options.setCapability("appium:platformVersion", "18.0"); options.setCapability("appium:deviceName", "iPhone 16"); HashMap tbOptions = new HashMap<>(); tbOptions.put("appiumVersion", "3.2.2"); options.setCapability("tb:options", tbOptions); ``` ```javascript const capabilities = { platformName: 'iOS', 'appium:platformVersion': '18.0', 'appium:deviceName': 'iPhone 16', 'appium:automationName': 'XCUITest', 'tb:options': { appiumVersion: '3.2.2' } } ``` ```python capabilities = { "platformName" : "iOS", "appium:platformVersion" : "18.0", "appium:deviceName" : "iPhone 16", "appium:automationName": "XCUITest", "tb:options" : { "appiumVersion" : "3.2.2" } } ``` ```ruby capabilities = { "platformName" => "iOS", "appium:platformVersion" => "18.0", "appium:deviceName" => "iPhone 16", "appium:automationName" => "XCUITest", "tb:options" => { "appiumVersion" => "3.2.2" } } ``` ```csharp AppiumOptions options = new AppiumOptions(); options.PlatformName = "iOS"; options.AddAdditionalAppiumOption("appium:platformVersion", "18.0"); options.AddAdditionalAppiumOption("appium:deviceName", "iPhone 16"); options.AddAdditionalAppiumOption("appium:automationName", "XCUITest"); var tbOptions = new Dictionary(); tbOptions.Add("appiumVersion", "3.2.2"); options.AddAdditionalAppiumOption("tb:options", tbOptions); ``` ## Appium 2 Changes If you've been using Appium 1 and want to try out Appium 2, then please be aware of the possible breaking changes listed below. ### Protocol Changes Appium 2 only allows the [W3C WebDriver Protocol](https://www.w3.org/TR/webdriver/). Even though Appium 1 has supported this protocol since recent years, it also supported JSONWP (JSON Wire Protocol) and MJSONWP (Mobile Json Wire Protocol). This means that if you are using an older Appium client or binding, you might have trouble communicating with Appium 2. Instead of using `desiredCapabilities`, you now need to use `capabilities` with either `alwaysMatch` or `firstMatch` properties. ### Capabilities Appium only accepts a couple of [standard W3C capabilities](https://www.w3.org/TR/webdriver/#capabilities), including: - `browserName` - `browserVersion` - `platformName` All other capabilities, such as `app` or `deviceName` need to be vendor-prefixed. A vendor prefix means you prefix each key with a string followed by a colon. In Appium's case, this would be `appium:` An example of some of the capabilities you can use with Appium 2: - `appium:app` - `appium:deviceName` - `appium:noReset` ### automationName is required With Appium 2, you now need to specify an automationName. Before, with Appium 1 this was filled in automatically if you did not specify it. For iOS testing, you should use `appium:automationName` with `XCUITest`. For Android, you should set the `appium:automationName` capability to `UiAutomator2`. ### TestingBot specific capabilities You can specify [TestingBot specific capabilities](https://testingbot.com/support/app-automate/appium/options) with the `tb:options` capability. See below for an example where we will be using both Appium 2 and custom TestingBot capabilities. ```java XCUITestOptions options = new XCUITestOptions(); options.setPlatformName("iOS"); options.setCapability("appium:platformVersion", "18.0"); options.setCapability("appium:deviceName", "iPhone 16"); HashMap tbOptions = new HashMap<>(); tbOptions.put("appiumVersion", "3.2.2"); options.setCapability("tb:options", tbOptions); ``` ## Frequently asked questions Which Appium version does TestingBot use by default? TestingBot selects the latest stable Appium release compatible with the device, OS and OS version you request, biased toward Appium 3 for current iOS and Android versions. To pin a specific version, set the `appiumVersion` capability inside `tb:options`. Use `latest`, `latest-1` or an explicit version number like `3.2.2`. How do I pin a specific Appium version for my test? Add `appiumVersion` inside the `tb:options` capability map. For example, in WebdriverIO: `'tb:options': { appiumVersion: '3.2.2' }`. See the [code example below](https://testingbot.com#example) for Java, Node.js, Python, Ruby and C# snippets. What is new in Appium 3 compared to Appium 2? Appium 3 removes a number of deprecated endpoints, bumps the minimum Node.js and NPM versions (handled by TestingBot), and introduces a security feature flag prefix (for example `adb_shell` becomes `uiautomator2:adb_shell`). See the [Appium 3 migration guide](https://appium.io/docs/en/latest/guides/migrating-2-to-3/) for the full list. What is the difference between Appium 1 and Appium 2? Appium 2 introduced the plugin and driver architecture, allowing platform support to be installed as separate drivers (xcuitest for iOS, uiautomator2 for Android, and so on). It also restricts the protocol to W3C WebDriver only, requires the appium: vendor prefix for non-standard capabilities, and requires automationName to be set explicitly. Appium 1 is kept as a legacy option for older test suites. Is Appium 1 still supported on TestingBot? Yes, Appium 1 is supported as a legacy option on the grid, currently at `1.22.3`. We recommend migrating to Appium 3 for new tests because Appium 1 receives no upstream updates. To stay on Appium 1, pin `appiumVersion: "1.22.3"` in your `tb:options`. Why is automationName required on Appium 2 and 3? Appium 2 and 3 use a driver architecture where each platform driver registers a name. You must tell Appium which driver to use by setting the `appium:automationName` capability. For iOS use `XCUITest`; for Android use `UiAutomator2`; for Flutter use `Flutter`. Can I use Appium 2 plugins on TestingBot? Yes. Activate plugins per-test with the `tb:appiumPlugins` capability. The Images, Execute Driver and Universal XML plugins are pre-installed on the grid. For example: `'tb:appiumPlugins': ['images', 'execute-driver', 'universal-xml']`. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/appium/csharp ### C# Examples: - [NUnit](https://testingbot.com/support/app-automate/appium/csharp/nunit) - [Reqnroll](https://testingbot.com/support/app-automate/appium/csharp/reqnroll) - [SpecFlow](https://testingbot.com/support/app-automate/appium/csharp/specflow) # C# Automated App Testing This guide will help you run mobile automated tests with Appium and C# on the TestingBot device cloud. [Command Line](https://testingbot.com#) [Visual Studio](https://testingbot.com#) TestingBot has created a sample Visual Studio project to help you get started with C# and Appium testing. [Download ZIP](https://github.com/testingbot/csharp-app-example/archive/refs/heads/main.zip) You can import this project into Visual Studio and run the tests directly. To run your tests with C# and .NET, you need to have the [.NET SDK](https://dotnet.microsoft.com/download) installed on your machine. - **On Windows:** - Download the [.NET SDK](https://dotnet.microsoft.com/en-us/download) installer and run it. - Or use the `winget` command: `winget install Microsoft.DotNet.SDK.9` - **On MacOS:** `brew install dotnet` - **On Linux:** `sudo apt-get install dotnet-sdk-9.0` After installing the .NET SDK, you can create a new NUnit test project using the command line: ```bash dotnet new nunit -n AppiumTest cd AppiumTest ``` Next, add the Appium and Selenium WebDriver packages to your project: ```bash dotnet add package Appium.WebDriver dotnet add package Selenium.WebDriver dotnet add package Selenium.Support ``` Replace the contents of `UnitTest1.cs` with the example code below, then run your tests with: ```bash TB_KEY=... TB_SECRET=... dotnet test ``` ## Installation See our [C# App example repository](https://github.com/testingbot/csharp-app-example) for a simple example on how to run C# tests on Mobile Devices with TestingBot. Add the [Appium.WebDriver](https://www.nuget.org/packages/Appium.WebDriver) NuGet package to your project. It provides the `OpenQA.Selenium.Appium` namespace and is the maintained, shipped .NET Appium client. You can use NuGet packages as well, in which case you'll need to install these packages: - Appium.WebDriver - DotNetSeleniumExtras.WaitHelpers - Selenium.WebDriver ## Real Device Testing [Android](https://testingbot.com#) [iOS](https://testingbot.com#) This will download and install a sample Android app we built. It will add two numbers to a textinput and display the sum of the two numbers. ```csharp using NUnit.Framework; using NUnit.Framework.Interfaces; using OpenQA.Selenium; using OpenQA.Selenium.Appium; using OpenQA.Selenium.Appium.Android; using OpenQA.Selenium.Support.UI; using System; using System.Collections.Generic; namespace AppiumNunit { [TestFixture] public class AndroidRealDeviceTest { private AndroidDriver? Driver; [Test] public void SampleTest() { var options = new AppiumOptions { PlatformName = "Android", AutomationName = "UiAutomator2", App = "https://testingbot.com/appium/sample.apk" }; options.AddAdditionalAppiumOption("appium:deviceName", "Galaxy S24"); options.AddAdditionalAppiumOption("appium:platformVersion", "14.0"); options.AddAdditionalAppiumOption("tb:options", new Dictionary { ["key"] = Environment.GetEnvironmentVariable("TESTINGBOT_KEY") ?? "", ["secret"] = Environment.GetEnvironmentVariable("TESTINGBOT_SECRET") ?? "", ["name"] = TestContext.CurrentContext.Test.Name, ["realDevice"] = true }); Driver = new AndroidDriver(new Uri("https://hub.testingbot.com/wd/hub"), options, TimeSpan.FromSeconds(120)); var wait = new WebDriverWait(Driver, TimeSpan.FromSeconds(30)); var inputA = wait.Until(d => d.FindElement(AppiumBy.AccessibilityId("inputA"))); inputA.SendKeys("10"); var inputB = wait.Until(d => d.FindElement(AppiumBy.AccessibilityId("inputB"))); inputB.SendKeys("5"); Assert.Pass(); } [TearDown] public void CleanUpAfterEveryTestMethod() { if (Driver != null) { var passed = TestContext.CurrentContext.Result.Outcome.Status == TestStatus.Passed; ((IJavaScriptExecutor)Driver).ExecuteScript("tb:test-result=" + (passed ? "passed" : "failed")); Driver.Dispose(); } } } } ``` This will download and install a sample iOS app we built. It will add two numbers to a textinput and display the sum of the two numbers. ```csharp using NUnit.Framework; using NUnit.Framework.Interfaces; using OpenQA.Selenium; using OpenQA.Selenium.Appium; using OpenQA.Selenium.Appium.iOS; using OpenQA.Selenium.Support.UI; using System; using System.Collections.Generic; namespace AppiumNunit { [TestFixture] public class iOSRealDeviceTest { private IOSDriver? Driver; [Test] public void SampleTest() { var options = new AppiumOptions { PlatformName = "iOS", AutomationName = "XCUITest", App = "https://testingbot.com/appium/sample.ipa" }; options.AddAdditionalAppiumOption("appium:deviceName", "iPhone 15"); options.AddAdditionalAppiumOption("appium:platformVersion", "18.0"); options.AddAdditionalAppiumOption("tb:options", new Dictionary { ["key"] = Environment.GetEnvironmentVariable("TESTINGBOT_KEY") ?? "", ["secret"] = Environment.GetEnvironmentVariable("TESTINGBOT_SECRET") ?? "", ["name"] = TestContext.CurrentContext.Test.Name, ["realDevice"] = true }); Driver = new IOSDriver(new Uri("https://hub.testingbot.com/wd/hub"), options, TimeSpan.FromSeconds(300)); var wait = new WebDriverWait(Driver, TimeSpan.FromSeconds(30)); var inputA = wait.Until(d => d.FindElement(AppiumBy.AccessibilityId("inputA"))); inputA.SendKeys("10"); var inputB = wait.Until(d => d.FindElement(AppiumBy.AccessibilityId("inputB"))); inputB.SendKeys("5"); var sum = wait.Until(d => d.FindElement(AppiumBy.AccessibilityId("sum"))); Console.WriteLine(sum.Text); Assert.Pass(); } [TearDown] public void CleanUpAfterEveryTestMethod() { if (Driver != null) { var passed = TestContext.CurrentContext.Result.Outcome.Status == TestStatus.Passed; ((IJavaScriptExecutor)Driver).ExecuteScript("tb:test-result=" + (passed ? "passed" : "failed")); Driver.Dispose(); } } } } ``` ## Simulator/Emulator Testing [Android](https://testingbot.com#) [iOS](https://testingbot.com#) This will download and install a sample Android app we built. It will add two numbers to a textinput and display the sum of the two numbers. ```csharp using NUnit.Framework; using NUnit.Framework.Interfaces; using OpenQA.Selenium; using OpenQA.Selenium.Appium; using OpenQA.Selenium.Appium.Android; using OpenQA.Selenium.Support.UI; using System; using System.Collections.Generic; namespace AppiumNunit { [TestFixture] public class AndroidEmulatorTest { private AndroidDriver? Driver; [Test] public void SampleTest() { var options = new AppiumOptions { PlatformName = "Android", AutomationName = "UiAutomator2", App = "https://testingbot.com/appium/sample.apk" }; options.AddAdditionalAppiumOption("appium:deviceName", "Pixel 8"); options.AddAdditionalAppiumOption("appium:platformVersion", "14.0"); options.AddAdditionalAppiumOption("tb:options", new Dictionary { ["key"] = Environment.GetEnvironmentVariable("TESTINGBOT_KEY") ?? "", ["secret"] = Environment.GetEnvironmentVariable("TESTINGBOT_SECRET") ?? "", ["name"] = TestContext.CurrentContext.Test.Name }); Driver = new AndroidDriver(new Uri("https://hub.testingbot.com/wd/hub"), options, TimeSpan.FromSeconds(240)); var wait = new WebDriverWait(Driver, TimeSpan.FromSeconds(30)); var inputA = wait.Until(d => d.FindElement(AppiumBy.AccessibilityId("inputA"))); inputA.SendKeys("10"); var inputB = wait.Until(d => d.FindElement(AppiumBy.AccessibilityId("inputB"))); inputB.SendKeys("5"); Assert.Pass(); } [TearDown] public void CleanUpAfterEveryTestMethod() { if (Driver != null) { var passed = TestContext.CurrentContext.Result.Outcome.Status == TestStatus.Passed; ((IJavaScriptExecutor)Driver).ExecuteScript("tb:test-result=" + (passed ? "passed" : "failed")); Driver.Dispose(); } } } } ``` This will download and install a sample iOS app we built. It will add two numbers to a textinput and display the sum of the two numbers. ```csharp using NUnit.Framework; using NUnit.Framework.Interfaces; using OpenQA.Selenium; using OpenQA.Selenium.Appium; using OpenQA.Selenium.Appium.iOS; using OpenQA.Selenium.Support.UI; using System; using System.Collections.Generic; namespace AppiumNunit { [TestFixture] public class iOSSimulatorTest { private IOSDriver? Driver; [Test] public void SampleTest() { var options = new AppiumOptions { PlatformName = "iOS", AutomationName = "XCUITest", App = "https://testingbot.com/appium/sample.zip" }; options.AddAdditionalAppiumOption("appium:deviceName", "iPhone 15"); options.AddAdditionalAppiumOption("appium:platformVersion", "17.4"); options.AddAdditionalAppiumOption("tb:options", new Dictionary { ["key"] = Environment.GetEnvironmentVariable("TESTINGBOT_KEY") ?? "", ["secret"] = Environment.GetEnvironmentVariable("TESTINGBOT_SECRET") ?? "", ["name"] = TestContext.CurrentContext.Test.Name }); Driver = new IOSDriver(new Uri("https://hub.testingbot.com/wd/hub"), options, TimeSpan.FromSeconds(300)); var wait = new WebDriverWait(Driver, TimeSpan.FromSeconds(30)); var inputA = wait.Until(d => d.FindElement(AppiumBy.AccessibilityId("inputA"))); inputA.SendKeys("10"); var inputB = wait.Until(d => d.FindElement(AppiumBy.AccessibilityId("inputB"))); inputB.SendKeys("5"); var sum = wait.Until(d => d.FindElement(AppiumBy.AccessibilityId("sum"))); Console.WriteLine(sum.Text); Assert.Pass(); } [TearDown] public void CleanUpAfterEveryTestMethod() { if (Driver != null) { var passed = TestContext.CurrentContext.Result.Outcome.Status == TestStatus.Passed; ((IJavaScriptExecutor)Driver).ExecuteScript("tb:test-result=" + (passed ? "passed" : "failed")); Driver.Dispose(); } } } } ``` ## Uploading Your App Please see our [Upload Mobile App](https://testingbot.com/support/app-automate/help/upload) documentation to find out how to upload your app to TestingBot for testing. ## Specify Browsers & Devices To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before:** ```csharp Driver = new IOSDriver(new Uri("http://localhost:4444/wd/hub"), options, TimeSpan.FromSeconds(300)); ``` **After:** ```csharp Driver = new IOSDriver(new Uri("https://hub.testingbot.com/wd/hub"), options, TimeSpan.FromSeconds(300)); ``` To let TestingBot know on which device you want to run your test on, you need to specify the `appium:deviceName`, `appium:platformVersion`, `platformName` and other optional options in the capabilities field. To see how to do this, please select a combination of device type and device name in the drop-down menus below. ![OS selected](https://testingbot.com/assets/environments/svg/ios-383468addf160fa18d0e431f529420739d7f7d1206175f682fead627d2e99a52.svg) iOS 18.6 › iPhone 16 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a C# test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```csharp using NUnit.Framework; using NUnit.Framework.Interfaces; using OpenQA.Selenium; using OpenQA.Selenium.Appium; using OpenQA.Selenium.Appium.iOS; using OpenQA.Selenium.Support.UI; using System; using System.Collections.Generic; namespace AppiumNunit { [TestFixture] public class iOSTunnelTest { private IOSDriver? Driver; [Test] public void SampleTest() { var options = new AppiumOptions { PlatformName = "iOS", AutomationName = "XCUITest", App = "https://testingbot.com/appium/sample.zip" }; options.AddAdditionalAppiumOption("appium:deviceName", "iPhone 15"); options.AddAdditionalAppiumOption("appium:platformVersion", "17.4"); options.AddAdditionalAppiumOption("tb:options", new Dictionary { ["name"] = TestContext.CurrentContext.Test.Name }); Driver = new IOSDriver(new Uri("http://localhost:4445/wd/hub"), options, TimeSpan.FromSeconds(300)); var wait = new WebDriverWait(Driver, TimeSpan.FromSeconds(30)); var inputA = wait.Until(d => d.FindElement(AppiumBy.AccessibilityId("inputA"))); inputA.SendKeys("10"); var inputB = wait.Until(d => d.FindElement(AppiumBy.AccessibilityId("inputB"))); inputB.SendKeys("5"); var sum = wait.Until(d => d.FindElement(AppiumBy.AccessibilityId("sum"))); Console.WriteLine(sum.Text); Assert.Pass(); } [TearDown] public void CleanUpAfterEveryTestMethod() { if (Driver != null) { var passed = TestContext.CurrentContext.Result.Outcome.Status == TestStatus.Passed; ((IJavaScriptExecutor)Driver).ExecuteScript("tb:test-result=" + (passed ? "passed" : "failed")); Driver.Dispose(); } } } } ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed To see if a test passed or not in our member area, or to send additional meta-data to TestingBot, you can use our API. Please see the example below on how to notify TestingBot about the test success state: ```csharp [TearDown] public void CleanUp() { if (Driver != null) { bool passed = TestContext.CurrentContext.Result.Outcome.Status == TestStatus.Passed; // Logs the result to TestingBot ((IJavaScriptExecutor)Driver).ExecuteScript("tb:test-result=" + (passed ? "passed" : "failed")); // Terminates the remote webdriver session Driver.Dispose(); } } ``` ## Preparing your App You do not need to install any code or plugin to run a test. Below are some things that are necessary to successfully run a mobile test: **For Android:** - Please supply the URL to your `.apk` or `.aab` file. **Important:** the `.apk` file needs to be a x86 build (x86 ABI) for Android emulators. **For iOS Real Device:** - Please supply the URL to an `.ipa` file. **For iOS Simulator:** - Please supply the URL to a `.zip` file that contains your `.app` - The `.app` needs to be an iOS Simulator build: - Create a Simulator build: ```bash xcodebuild -sdk iphonesimulator -configuration Debug ``` - Zip the `.app` bundle: ```bash zip -r MyApp.zip MyApp.app ``` ## Additional Options Appium provides [a lot of options](https://appium.io/docs/en/2.0/guides/caps/) to configure your test. Some important options that might help: **For Android:** - **appActivity** and **appPackage** : by default, Appium will try to extract the main Activity from your apk. If this fails, you can supply your own with these options. - **chromeOptions** : additional chromedriver options you can supply. - **otherApps** : a JSON array of other apps that need to be installed, in URL format. **For Android & iOS:** - **locale** : the language of the simulator/device (ex: `fr_CA`) This sets the locale on the entire device/simulator, except on physical iOS devices. For real iOS devices, we will pass `-AppleLocale` as argument to the app. - **newCommandTimeout** : How long (in seconds) Appium will wait for a new command from the client before assuming the client quit and ending the session ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/appium/csharp/specflow # SpecFlow Automated App Testing SpecFlow reached end-of-life at the end of 2024 and is succeeded by [Reqnroll](https://reqnroll.net). For new projects we recommend using [Reqnroll](https://testingbot.com/support/app-automate/appium/csharp/reqnroll) instead. The examples below still work with SpecFlow 3.x but new development should target Reqnroll. See our [SpecFlow App example repository](https://github.com/testingbot/specflow-app-example) for a simple example on how to run SpecFlow tests on Mobile Devices with TestingBot. This example project can be used with Visual Studio, or via terminal: `dotnet test`. SpecFlow is an open-source .NET utility which allows you to write tests using Cucumber-compatible Gherkin syntax. ## Installation We'll assume you are using Visual Studio to run your tests. Here are the steps to set up the required environment for your SpecFlow tests: - Add NuGet Packages: `Appium.WebDriver, NUnit, SpecFlow` - For more information, please see the [SpecFlow documentation](https://specflow.org/getting-started/#InstallSetup). In the example below, we'll create a calculator test that enters 2 numbers in 2 input fields and validates the sum. The app used in the example is a demo app we created at TestingBot and runs on both iOS and Android. ## SpecFlow Example First, we'll create the feature file **feature.feature** : ```gherkin Feature: Calculator Scenario Outline: Can do simple math Given I am using the calculator When I add inputA for "5" When I add inputB for "10" Then I should see the sum "15" ``` Now we need to define the logic behind these steps: **SingleStep.cs** ```csharp using NUnit.Framework; using OpenQA.Selenium.Appium; using OpenQA.Selenium.Appium.Android; using TechTalk.SpecFlow; namespace testingbot_specflow { [Binding] public class SingleStep { private readonly ScenarioContext _scenarioContext; private AndroidDriver _driver; private readonly TestingBotDriver _tbDriver; public SingleStep(ScenarioContext scenarioContext) { _scenarioContext = scenarioContext; _tbDriver = (TestingBotDriver) _scenarioContext["tbDriver"]; } [Given(@"I am using the calculator")] public void GivenIAmOnTheGooglePage() { _driver = _tbDriver.Init(); // The app launches automatically because appium:app is set in the capabilities. // If you need to relaunch it explicitly, use the typed convenience method: // _driver.ActivateApp(""); } [When(@"I add inputA for ""(.*)""")] public void WhenIAddA(string amount) { var inputA = _driver.FindElement(AppiumBy.Id("inputA")); inputA.SendKeys(amount); } [When(@"I add inputB for ""(.*)""")] public void WhenIAddB(string amount) { var inputB = _driver.FindElement(AppiumBy.Id("inputB")); inputB.SendKeys(amount); } [Then(@"I should see the sum ""(.*)""")] public void ThenIShouldSeeTitle(string sum) { Assert.That(_driver.FindElement(AppiumBy.XPath("//android.widget.EditText[@content-desc=\"sum\"]")).Text, Is.EqualTo(sum)); } } } ``` Finally, we'll need to create the `TestingBotDriver`: ```csharp using System; using OpenQA.Selenium; using OpenQA.Selenium.Appium; using OpenQA.Selenium.Appium.Android; using OpenQA.Selenium.Appium.Enums; using TechTalk.SpecFlow; namespace testingbot_specflow { [Binding] public sealed class testingbot_specflow { private readonly ScenarioContext _scenarioContext; private TestingBotDriver tbDriver; public testingbot_specflow(ScenarioContext scenarioContext) { _scenarioContext = scenarioContext; } [BeforeScenario] public void BeforeScenario() { tbDriver = new TestingBotDriver(_scenarioContext); _scenarioContext["tbDriver"] = tbDriver; } [AfterScenario] public void AfterScenario() { tbDriver.Cleanup(); } } public class TestingBotDriver { private AndroidDriver driver; private ScenarioContext current; public TestingBotDriver(ScenarioContext current) { this.current = current; } public AndroidDriver Init() { AppiumOptions appiumOptions = new AppiumOptions { PlatformName = "Android", AutomationName = "UiAutomator2" }; appiumOptions.AddAdditionalAppiumOption("appium:deviceName", "Galaxy S9"); appiumOptions.AddAdditionalAppiumOption("appium:platformVersion", "9.0"); appiumOptions.AddAdditionalAppiumOption("appium:app", "tb://..."); string key = "TESTINGBOT_KEY"; string secret = "TESTINGBOT_SECRET"; appiumOptions.AddAdditionalAppiumOption("tb:options", new Dictionary { ["key"] = key, ["secret"] = secret }); AndroidDriver driver = new AndroidDriver( new Uri("https://hub.testingbot.com/wd/hub"), appiumOptions ); return driver; } public void Cleanup() { if (driver != null) { driver.Quit(); } } } } ``` ## Real Device Testing [Android](https://testingbot.com#) [iOS](https://testingbot.com#) This will download and install a sample Android app we built. It will add two numbers to a textinput and display the sum of the two numbers. ```csharp using System; using OpenQA.Selenium; using OpenQA.Selenium.Appium; using OpenQA.Selenium.Appium.Android; using OpenQA.Selenium.Appium.Enums; using TechTalk.SpecFlow; namespace testingbot_specflow { [Binding] public sealed class testingbot_specflow { private readonly ScenarioContext _scenarioContext; private TestingBotDriver tbDriver; public testingbot_specflow(ScenarioContext scenarioContext) { _scenarioContext = scenarioContext; } [BeforeScenario] public void BeforeScenario() { tbDriver = new TestingBotDriver(_scenarioContext); _scenarioContext["tbDriver"] = tbDriver; } [AfterScenario] public void AfterScenario() { tbDriver.Cleanup(); } } public class TestingBotDriver { private AndroidDriver driver; private ScenarioContext current; public TestingBotDriver(ScenarioContext current) { this.current = current; } public AndroidDriver Init() { AppiumOptions appiumOptions = new AppiumOptions { PlatformName = "Android", AutomationName = "UiAutomator2" }; appiumOptions.AddAdditionalAppiumOption("appium:deviceName", "Galaxy S10"); appiumOptions.AddAdditionalAppiumOption("appium:platformVersion", "10.0"); appiumOptions.AddAdditionalAppiumOption("appium:app", "tb://..."); appiumOptions.AddAdditionalAppiumOption("tb:options", new Dictionary { ["key"] = Environment.GetEnvironmentVariable("TESTINGBOT_KEY"), ["secret"] = Environment.GetEnvironmentVariable("TESTINGBOT_SECRET"), ["realDevice"] = true }); AndroidDriver driver = new AndroidDriver( new Uri("https://hub.testingbot.com/wd/hub"), appiumOptions ); return driver; } public void Cleanup() { if (driver != null) { driver.Quit(); } } } } ``` This will download and install a sample iOS app we built. It will add two numbers to a textinput and display the sum of the two numbers. ```csharp using System; using OpenQA.Selenium.Appium; using OpenQA.Selenium.Appium.Enums; using OpenQA.Selenium.Appium.iOS; using TechTalk.SpecFlow; namespace testingbot_specflow { [Binding] public sealed class testingbot_specflow { private readonly ScenarioContext _scenarioContext; private TestingBotDriver tbDriver; public testingbot_specflow(ScenarioContext scenarioContext) { _scenarioContext = scenarioContext; } [BeforeScenario] public void BeforeScenario() { tbDriver = new TestingBotDriver(_scenarioContext); _scenarioContext["tbDriver"] = tbDriver; } [AfterScenario] public void AfterScenario() { tbDriver.Cleanup(); } } public class TestingBotDriver { private IOSDriver driver; private ScenarioContext current; public TestingBotDriver(ScenarioContext current) { this.current = current; } public IOSDriver Init() { AppiumOptions appiumOptions = new AppiumOptions { PlatformName = "iOS", AutomationName = "XCUITest" }; appiumOptions.AddAdditionalAppiumOption("appium:deviceName", "iPhone 15"); appiumOptions.AddAdditionalAppiumOption("appium:platformVersion", "18.0"); appiumOptions.AddAdditionalAppiumOption("appium:app", "tb://..."); appiumOptions.AddAdditionalAppiumOption("tb:options", new Dictionary { ["key"] = Environment.GetEnvironmentVariable("TESTINGBOT_KEY"), ["secret"] = Environment.GetEnvironmentVariable("TESTINGBOT_SECRET"), ["realDevice"] = true }); IOSDriver driver = new IOSDriver( new Uri("https://hub.testingbot.com/wd/hub"), appiumOptions ); return driver; } public void Cleanup() { if (driver != null) { driver.Quit(); } } } } ``` ## Simulator/Emulator Testing [Android](https://testingbot.com#) [iOS](https://testingbot.com#) This will download and install a sample Android app we built. It will add two numbers to a textinput and display the sum of the two numbers. ```csharp using System; using OpenQA.Selenium; using OpenQA.Selenium.Appium; using OpenQA.Selenium.Appium.Android; using OpenQA.Selenium.Appium.Enums; using TechTalk.SpecFlow; namespace testingbot_specflow { [Binding] public sealed class testingbot_specflow { private readonly ScenarioContext _scenarioContext; private TestingBotDriver tbDriver; public testingbot_specflow(ScenarioContext scenarioContext) { _scenarioContext = scenarioContext; } [BeforeScenario] public void BeforeScenario() { tbDriver = new TestingBotDriver(_scenarioContext); _scenarioContext["tbDriver"] = tbDriver; } [AfterScenario] public void AfterScenario() { tbDriver.Cleanup(); } } public class TestingBotDriver { private AndroidDriver driver; private ScenarioContext current; public TestingBotDriver(ScenarioContext current) { this.current = current; } public AndroidDriver Init() { AppiumOptions appiumOptions = new AppiumOptions { PlatformName = "Android", AutomationName = "UiAutomator2" }; appiumOptions.AddAdditionalAppiumOption("appium:deviceName", "Galaxy S9"); appiumOptions.AddAdditionalAppiumOption("appium:platformVersion", "9.0"); appiumOptions.AddAdditionalAppiumOption("appium:app", "tb://..."); appiumOptions.AddAdditionalAppiumOption("tb:options", new Dictionary { ["key"] = Environment.GetEnvironmentVariable("TESTINGBOT_KEY"), ["secret"] = Environment.GetEnvironmentVariable("TESTINGBOT_SECRET") }); AndroidDriver driver = new AndroidDriver( new Uri("https://hub.testingbot.com/wd/hub"), appiumOptions ); return driver; } public void Cleanup() { if (driver != null) { driver.Quit(); } } } } ``` This will download and install a sample iOS app we built. It will add two numbers to a textinput and display the sum of the two numbers. ```csharp using System; using OpenQA.Selenium.Appium; using OpenQA.Selenium.Appium.Enums; using OpenQA.Selenium.Appium.iOS; using TechTalk.SpecFlow; namespace testingbot_specflow { [Binding] public sealed class testingbot_specflow { private readonly ScenarioContext _scenarioContext; private TestingBotDriver tbDriver; public testingbot_specflow(ScenarioContext scenarioContext) { _scenarioContext = scenarioContext; } [BeforeScenario] public void BeforeScenario() { tbDriver = new TestingBotDriver(_scenarioContext); _scenarioContext["tbDriver"] = tbDriver; } [AfterScenario] public void AfterScenario() { tbDriver.Cleanup(); } } public class TestingBotDriver { private IOSDriver driver; private ScenarioContext current; public TestingBotDriver(ScenarioContext current) { this.current = current; } public IOSDriver Init() { AppiumOptions appiumOptions = new AppiumOptions { PlatformName = "iOS", AutomationName = "XCUITest" }; appiumOptions.AddAdditionalAppiumOption("appium:deviceName", "iPhone 16"); appiumOptions.AddAdditionalAppiumOption("appium:platformVersion", "18.0"); appiumOptions.AddAdditionalAppiumOption("appium:app", "tb://..."); appiumOptions.AddAdditionalAppiumOption("tb:options", new Dictionary { ["key"] = Environment.GetEnvironmentVariable("TESTINGBOT_KEY"), ["secret"] = Environment.GetEnvironmentVariable("TESTINGBOT_SECRET") }); IOSDriver driver = new IOSDriver( new Uri("https://hub.testingbot.com/wd/hub"), appiumOptions ); return driver; } public void Cleanup() { if (driver != null) { driver.Quit(); } } } } ``` ## Uploading Your App Please see our [Upload Mobile App](https://testingbot.com/support/app-automate/help/upload) documentation to find out how to upload your app to TestingBot for testing. ## Specify Browsers & Devices To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before:** ```csharp IOSDriver driver = new IOSDriver( new Uri("http://localhost:4723"), appiumOptions ); ``` **After:** ```csharp IOSDriver driver = new IOSDriver( new Uri("https://hub.testingbot.com/wd/hub"), appiumOptions ); ``` ### Configuring Capabilities Capabilities are key-value pairs that allows you to customize your tests on TestingBot. Appium provides its [own set of capabilities](https://appium.io/docs/en/2.0/guides/caps/) which you can specify. TestingBot also provide its own [Custom Capabilities](https://testingbot.com/support/app-automate/appium/options), to customize tests ran on the TestingBot platform. You can use the drop-down menus below to see how to configure your tests to run on a specific mobile device: ![OS selected](https://testingbot.com/assets/environments/svg/ios-383468addf160fa18d0e431f529420739d7f7d1206175f682fead627d2e99a52.svg) iOS 18.6 › iPhone 16 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a C# test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```csharp using System; using System.Collections.Generic; using OpenQA.Selenium.Appium; using OpenQA.Selenium.Appium.Android; namespace AppiumTunnelTest { class Program { static void Main(string[] args) { var appiumOptions = new AppiumOptions { PlatformName = "Android", AutomationName = "UiAutomator2", App = "tb://..." }; appiumOptions.AddAdditionalAppiumOption("appium:deviceName", "Pixel 8"); appiumOptions.AddAdditionalAppiumOption("appium:platformVersion", "14.0"); appiumOptions.AddAdditionalAppiumOption("tb:options", new Dictionary { ["name"] = "SpecFlow Tunnel Test" }); // Point to local tunnel instead of hub.testingbot.com var driver = new AndroidDriver( new Uri("http://localhost:4445/wd/hub"), appiumOptions ); // Run your app test against the internal/staged app... driver.Quit(); } } } ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed To see if a test passed or not in our member area, or to send additional meta-data to TestingBot, you can use our API. Please see the example below on how to notify TestingBot about the test success state: ```csharp [TearDown] public void CleanUp() { bool passed = TestContext.CurrentContext.Result.Status == TestStatus.Passed; try { // Logs the result to TestingBot ((IJavaScriptExecutor)driver).ExecuteScript("tb:test-result=" + (passed ? "passed" : "failed")); } finally { // Terminates the remote webdriver session driver.Quit(); } } ``` ## Preparing your App You do not need to install any code or plugin to run a test. Below are some things that are necessary to successfully run a mobile test: **For Android:** - Please supply the URL to your `.apk` or `.aab` file. **Important:** the `.apk` file needs to be a x86 build (x86 ABI) for Android emulators. **For iOS Real Device:** - Please supply the URL to an `.ipa` file. **For iOS Simulator:** - Please supply the URL to a `.zip` file that contains your `.app` - The `.app` needs to be an iOS Simulator build: - Create a Simulator build: ```bash xcodebuild -sdk iphonesimulator -configuration Debug ``` - Zip the `.app` bundle: ```bash zip -r MyApp.zip MyApp.app ``` ## Additional Options Appium provides [a lot of options](https://appium.io/docs/en/2.0/guides/caps/) to configure your test. Some important options that might help: **For Android:** - **appActivity** and **appPackage** : by default, Appium will try to extract the main Activity from your apk. If this fails, you can supply your own with these options. - **chromeOptions** : additional chromedriver options you can supply. - **otherApps** : a JSON array of other apps that need to be installed, in URL format. **For Android & iOS:** - **locale** : the language of the simulator/device (ex: `fr_CA`) This sets the locale on the entire device/simulator, except on physical iOS devices. For real iOS devices, we will pass `-AppleLocale` as argument to the app. - **newCommandTimeout** : How long (in seconds) Appium will wait for a new command from the client before assuming the client quit and ending the session ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/appium/csharp/reqnroll # Reqnroll Automated App Testing [Reqnroll](https://reqnroll.net/) is an open-source .NET test automation framework for Behavior-Driven Development (BDD). It is the community-maintained successor to SpecFlow and is fully compatible with Gherkin syntax. Reqnroll works with all major .NET versions (.NET 6.0, 7.0, 8.0, 9.0) and integrates with popular test frameworks: - NUnit - xUnit - MSTest - TUnit **Migrating from SpecFlow?** Reqnroll is based on the SpecFlow codebase and provides an easy migration path. See the [migration section](https://testingbot.com#migration) below. In the example below, we run a Gherkin scenario against the TestingBot demo app on both Android and iOS. The app adds two numbers and shows the sum. ## Installation First, install the **Reqnroll for Visual Studio 2022** extension from the Visual Studio Marketplace to get syntax highlighting, navigation and project templates. ### Create a new project Create a new test project and add the required packages: ```bash # Create a new NUnit test project dotnet new nunit -n AppiumReqnrollTests cd AppiumReqnrollTests # Add Reqnroll with NUnit integration dotnet add package Reqnroll.NUnit # Add the Appium .NET client dotnet add package Appium.WebDriver dotnet add package Selenium.WebDriver dotnet add package Selenium.Support ``` Or add these packages to your `.csproj` file: ```xml ``` Alternative packages for other test frameworks: - `Reqnroll.xUnit` - for xUnit 2.x - `Reqnroll.xUnit.v3` - for xUnit 3.x - `Reqnroll.MsTest` - for MSTest - `Reqnroll.TUnit` - for TUnit ## Your first Reqnroll test Reqnroll tests consist of three parts: a Feature file (Gherkin), Step Definitions (C#), and a Driver helper class. ### 1. Feature File Create a `Features` folder and add `Calculator.feature`: ```gherkin Feature: Calculator Scenario: Can do simple math Given I am using the calculator When I add inputA for "5" When I add inputB for "10" Then I should see the sum "15" ``` ### 2. Driver Helper Class Create a helper class to manage the Appium connection to TestingBot: ```csharp using OpenQA.Selenium; using OpenQA.Selenium.Appium; using OpenQA.Selenium.Appium.Android; namespace AppiumReqnrollTests.Support; public class TestingBotDriver : IDisposable { public AndroidDriver Driver { get; private set; } private bool _testPassed = true; public void Initialize() { var options = new AppiumOptions { PlatformName = "Android", AutomationName = "UiAutomator2", App = "https://testingbot.com/appium/sample.apk" }; options.AddAdditionalAppiumOption("appium:deviceName", "Galaxy S24"); options.AddAdditionalAppiumOption("appium:platformVersion", "14.0"); options.AddAdditionalAppiumOption("tb:options", new Dictionary { ["key"] = Environment.GetEnvironmentVariable("TB_KEY") ?? "", ["secret"] = Environment.GetEnvironmentVariable("TB_SECRET") ?? "", ["name"] = "Reqnroll Appium Test", ["realDevice"] = true }); Driver = new AndroidDriver( new Uri("https://hub.testingbot.com/wd/hub"), options, TimeSpan.FromSeconds(120) ); } public void MarkTestFailed() => _testPassed = false; public void Dispose() { if (Driver != null) { var status = _testPassed ? "passed" : "failed"; ((IJavaScriptExecutor)Driver).ExecuteScript($"tb:test-result={status}"); Driver.Dispose(); Driver = null; // prevents re-entry on the second Dispose } } } ``` ### 3. Step Definitions Create `StepDefinitions/CalculatorSteps.cs`: ```csharp using OpenQA.Selenium.Appium; using Reqnroll; using AppiumReqnrollTests.Support; namespace AppiumReqnrollTests.StepDefinitions; [Binding] public class CalculatorSteps { private readonly TestingBotDriver _tbDriver; public CalculatorSteps(TestingBotDriver tbDriver) { _tbDriver = tbDriver; } [Given("I am using the calculator")] public void GivenIAmUsingTheCalculator() { if (_tbDriver.Driver == null) _tbDriver.Initialize(); } [When(@"I add inputA for ""(.*)""")] public void WhenIAddInputA(string amount) { var inputA = _tbDriver.Driver.FindElement(AppiumBy.AccessibilityId("inputA")); inputA.SendKeys(amount); } [When(@"I add inputB for ""(.*)""")] public void WhenIAddInputB(string amount) { var inputB = _tbDriver.Driver.FindElement(AppiumBy.AccessibilityId("inputB")); inputB.SendKeys(amount); } [Then(@"I should see the sum ""(.*)""")] public void ThenIShouldSeeTheSum(string sum) { var sumElement = _tbDriver.Driver.FindElement(AppiumBy.AccessibilityId("sum")); Assert.That(sumElement.Text, Is.EqualTo(sum)); } } ``` ### 4. Hooks for Setup/Teardown Create `Hooks/TestHooks.cs` to manage the driver lifecycle: ```csharp using Reqnroll; using AppiumReqnrollTests.Support; namespace AppiumReqnrollTests.Hooks; [Binding] public class TestHooks { private readonly TestingBotDriver _tbDriver; public TestHooks(TestingBotDriver tbDriver) { _tbDriver = tbDriver; } [AfterScenario] public void AfterScenario(ScenarioContext scenarioContext) { if (scenarioContext.TestError != null) { _tbDriver.MarkTestFailed(); } _tbDriver.Dispose(); } } ``` Run your tests with: ```bash TB_KEY=your_key TB_SECRET=your_secret dotnet test ``` ## Real Device Testing [Android](https://testingbot.com#) [iOS](https://testingbot.com#) The example below runs the Reqnroll scenario on a real Android device. Set `realDevice: true` inside `tb:options` to target physical hardware. ```csharp using OpenQA.Selenium; using OpenQA.Selenium.Appium; using OpenQA.Selenium.Appium.Android; namespace AppiumReqnrollTests.Support; public class TestingBotDriver : IDisposable { public AndroidDriver Driver { get; private set; } private bool _testPassed = true; public void Initialize() { var options = new AppiumOptions { PlatformName = "Android", AutomationName = "UiAutomator2", App = "https://testingbot.com/appium/sample.apk" }; options.AddAdditionalAppiumOption("appium:deviceName", "Galaxy S24"); options.AddAdditionalAppiumOption("appium:platformVersion", "14.0"); options.AddAdditionalAppiumOption("tb:options", new Dictionary { ["key"] = Environment.GetEnvironmentVariable("TB_KEY") ?? "", ["secret"] = Environment.GetEnvironmentVariable("TB_SECRET") ?? "", ["name"] = "Reqnroll Android Real Device", ["realDevice"] = true }); Driver = new AndroidDriver( new Uri("https://hub.testingbot.com/wd/hub"), options, TimeSpan.FromSeconds(120) ); } public void MarkTestFailed() => _testPassed = false; public void Dispose() { if (Driver != null) { var status = _testPassed ? "passed" : "failed"; ((IJavaScriptExecutor)Driver).ExecuteScript($"tb:test-result={status}"); Driver.Dispose(); Driver = null; // prevents re-entry on the second Dispose } } } ``` The example below runs the Reqnroll scenario on a real iPhone. Use `IOSDriver` instead of `AndroidDriver`. ```csharp using OpenQA.Selenium; using OpenQA.Selenium.Appium; using OpenQA.Selenium.Appium.iOS; namespace AppiumReqnrollTests.Support; public class TestingBotDriver : IDisposable { public IOSDriver Driver { get; private set; } private bool _testPassed = true; public void Initialize() { var options = new AppiumOptions { PlatformName = "iOS", AutomationName = "XCUITest", App = "https://testingbot.com/appium/sample.ipa" }; options.AddAdditionalAppiumOption("appium:deviceName", "iPhone 15"); options.AddAdditionalAppiumOption("appium:platformVersion", "18.0"); options.AddAdditionalAppiumOption("tb:options", new Dictionary { ["key"] = Environment.GetEnvironmentVariable("TB_KEY") ?? "", ["secret"] = Environment.GetEnvironmentVariable("TB_SECRET") ?? "", ["name"] = "Reqnroll iOS Real Device", ["realDevice"] = true }); Driver = new IOSDriver( new Uri("https://hub.testingbot.com/wd/hub"), options, TimeSpan.FromSeconds(300) ); } public void MarkTestFailed() => _testPassed = false; public void Dispose() { if (Driver != null) { var status = _testPassed ? "passed" : "failed"; ((IJavaScriptExecutor)Driver).ExecuteScript($"tb:test-result={status}"); Driver.Dispose(); Driver = null; // prevents re-entry on the second Dispose } } } ``` ## Simulator/Emulator Testing [Android](https://testingbot.com#) [iOS](https://testingbot.com#) Run the same Reqnroll scenario on an Android emulator. Omit `realDevice` from `tb:options` to target an emulator. ```csharp var options = new AppiumOptions { PlatformName = "Android", AutomationName = "UiAutomator2", App = "https://testingbot.com/appium/sample.apk" }; options.AddAdditionalAppiumOption("appium:deviceName", "Pixel 8"); options.AddAdditionalAppiumOption("appium:platformVersion", "14.0"); options.AddAdditionalAppiumOption("tb:options", new Dictionary { ["key"] = Environment.GetEnvironmentVariable("TB_KEY") ?? "", ["secret"] = Environment.GetEnvironmentVariable("TB_SECRET") ?? "", ["name"] = "Reqnroll Android Emulator" }); Driver = new AndroidDriver( new Uri("https://hub.testingbot.com/wd/hub"), options, TimeSpan.FromSeconds(240) ); ``` Run the same Reqnroll scenario on an iOS simulator. ```csharp var options = new AppiumOptions { PlatformName = "iOS", AutomationName = "XCUITest", App = "https://testingbot.com/appium/sample.zip" }; options.AddAdditionalAppiumOption("appium:deviceName", "iPhone 15"); options.AddAdditionalAppiumOption("appium:platformVersion", "17.4"); options.AddAdditionalAppiumOption("tb:options", new Dictionary { ["key"] = Environment.GetEnvironmentVariable("TB_KEY") ?? "", ["secret"] = Environment.GetEnvironmentVariable("TB_SECRET") ?? "", ["name"] = "Reqnroll iOS Simulator" }); Driver = new IOSDriver( new Uri("https://hub.testingbot.com/wd/hub"), options, TimeSpan.FromSeconds(300) ); ``` ## Uploading Your App Please see our [Upload Mobile App](https://testingbot.com/support/app-automate/help/upload) documentation to find out how to upload your app to TestingBot for testing. ## Specify Browsers & Devices To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before:** ```csharp Driver = new IOSDriver( new Uri("http://localhost:4723/wd/hub"), options ); ``` **After:** ```csharp Driver = new IOSDriver( new Uri("https://hub.testingbot.com/wd/hub"), options ); ``` ### Configuring Capabilities Capabilities are key-value pairs that allow you to customize your tests on TestingBot. Appium provides its [own set of capabilities](https://appium.io/docs/en/2.0/guides/caps/) which you can specify. TestingBot also provides its own [Custom Capabilities](https://testingbot.com/support/app-automate/appium/options), to customize tests run on the TestingBot platform. You can use the drop-down menus below to see how to configure your tests to run on a specific mobile device: ![OS selected](https://testingbot.com/assets/environments/svg/ios-383468addf160fa18d0e431f529420739d7f7d1206175f682fead627d2e99a52.svg) iOS 18.6 › iPhone 16 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged or internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a Reqnroll test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above, change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```csharp var options = new AppiumOptions { PlatformName = "Android", AutomationName = "UiAutomator2", App = "tb://..." }; options.AddAdditionalAppiumOption("appium:deviceName", "Pixel 8"); options.AddAdditionalAppiumOption("appium:platformVersion", "14.0"); options.AddAdditionalAppiumOption("tb:options", new Dictionary { ["name"] = "Reqnroll Tunnel Test" }); // Point to the local tunnel instead of hub.testingbot.com Driver = new AndroidDriver( new Uri("http://localhost:4445/wd/hub"), options ); ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. Reqnroll supports parallel test execution when using NUnit or xUnit as the test runner. ### NUnit Parallel Execution Add to your `AssemblyInfo.cs` or any file: ```csharp using NUnit.Framework; [assembly: Parallelizable(ParallelScope.Fixtures)] [assembly: LevelOfParallelism(5)] ``` Configure in `reqnroll.json`: ```json { "$schema": "https://schemas.reqnroll.net/reqnroll-config-latest.json", "bindingCulture": { "name": "en-US" } } ``` ### xUnit Parallel Execution Add `xunit.runner.json`: ```json { "parallelizeAssembly": true, "parallelizeTestCollections": true, "maxParallelThreads": 5 } ``` ### Queuing Every TestingBot plan has a limit on parallel tests. If you exceed this limit, additional tests are queued (up to 6 minutes) and run as slots become available. ## Mark tests as passed/failed Report test results to TestingBot using JavaScript execution in your `[AfterScenario]` hook: ```csharp [Binding] public class TestHooks { private readonly TestingBotDriver _tbDriver; public TestHooks(TestingBotDriver tbDriver) { _tbDriver = tbDriver; } [AfterScenario] public void AfterScenario(ScenarioContext scenarioContext) { if (_tbDriver.Driver != null) { var status = scenarioContext.TestError == null ? "passed" : "failed"; ((IJavaScriptExecutor)_tbDriver.Driver).ExecuteScript($"tb:test-result={status}"); _tbDriver.Driver.Dispose(); } } } ``` ## Migrating from SpecFlow Reqnroll is designed as a drop-in replacement for SpecFlow. Most Appium projects can migrate with minimal changes: ### 1. Update NuGet Packages ```bash # Remove SpecFlow packages dotnet remove package SpecFlow dotnet remove package SpecFlow.NUnit # Add Reqnroll packages dotnet add package Reqnroll.NUnit ``` ### 2. Update Namespaces Replace `TechTalk.SpecFlow` with `Reqnroll` in your code: ```csharp // Before (SpecFlow) using TechTalk.SpecFlow; // After (Reqnroll) using Reqnroll; ``` ### 3. Update Configuration Rename `specflow.json` to `reqnroll.json` and update the schema reference: ```json { "$schema": "https://schemas.reqnroll.net/reqnroll-config-latest.json" } ``` ### 4. Update Visual Studio Extension Install the "Reqnroll for Visual Studio 2022" extension and uninstall the SpecFlow extension. For a complete migration guide, see the [official SpecFlow to Reqnroll migration guide](https://reqnroll.net/news/2024/02/from-specflow-to-reqnroll-why-and-how/). ## Preparing your App You do not need to install any code or plugin to run a test. Below are some things that are necessary to successfully run a mobile test: **For Android:** - Please supply the URL to your `.apk` or `.aab` file. **Important:** the `.apk` file needs to be a x86 build (x86 ABI) for Android emulators. **For iOS Real Device:** - Please supply the URL to an `.ipa` file. **For iOS Simulator:** - Please supply the URL to a `.zip` file that contains your `.app` - The `.app` needs to be an iOS Simulator build: - Create a Simulator build: ```bash xcodebuild -sdk iphonesimulator -configuration Debug ``` - Zip the `.app` bundle: ```bash zip -r MyApp.zip MyApp.app ``` ## Additional Options Appium provides [a lot of options](https://appium.io/docs/en/2.0/guides/caps/) to configure your test. Some important options that might help: **For Android:** - **appActivity** and **appPackage** : by default, Appium will try to extract the main Activity from your apk. If this fails, you can supply your own with these options. - **chromeOptions** : additional chromedriver options you can supply. - **otherApps** : a JSON array of other apps that need to be installed, in URL format. **For Android & iOS:** - **locale** : the language of the simulator/device (ex: `fr_CA`) This sets the locale on the entire device/simulator, except on physical iOS devices. For real iOS devices, we will pass `-AppleLocale` as argument to the app. - **newCommandTimeout** : How long (in seconds) Appium will wait for a new command from the client before assuming the client quit and ending the session ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/appium/csharp/nunit # NUnit Automated App Testing NUnit is a unit-testing framework for all .NET languages. You can use NUnit to run tests on mobile devices, together with Appium. This documentation page will show you how to set up and run Appium tests with NUnit on TestingBot's remote physical device grid. Once you've finished integrating, you will be able to run tests on iOS and Android devices, in parallel, on the remote TestingBot device grid. ## Installation We'll assume you are using Visual Studio to run your tests. Make sure to add these NuGet packages: - `Appium.WebDriver` - `NUnit` ## Initialize remote Appium Webdriver To be able to connect to TestingBot's device grid, we'll need to set up a remote Appium WebDriver. Please see the example file (`TestingBotDriver`) below on how to do this. [Android](https://testingbot.com#) [iOS](https://testingbot.com#) ```csharp using System; using System.Collections.Generic; using NUnit.Framework; using OpenQA.Selenium; using OpenQA.Selenium.Appium; using OpenQA.Selenium.Appium.Android; namespace testingbot { [TestFixture] public class TestingBotAndroidTest { private AndroidDriver driver; [SetUp] public void SetUp() { AppiumOptions appiumOptions = new AppiumOptions(); appiumOptions.PlatformName = "Android"; appiumOptions.AutomationName = "UiAutomator2"; appiumOptions.AddAdditionalAppiumOption("appium:deviceName", "Galaxy S23"); appiumOptions.AddAdditionalAppiumOption("appium:platformVersion", "13.0"); appiumOptions.AddAdditionalAppiumOption("appium:app", "tb://..."); string key = "TESTINGBOT_KEY"; string secret = "TESTINGBOT_SECRET"; var tbOptions = new Dictionary { ["key"] = key, ["secret"] = secret, ["realDevice"] = true }; appiumOptions.AddAdditionalAppiumOption("tb:options", tbOptions); driver = new AndroidDriver( new Uri("https://hub.testingbot.com/wd/hub"), appiumOptions ); } [Test] public void SampleTest() { driver.FindElement(AppiumBy.AccessibilityId("inputA")).SendKeys("10"); driver.FindElement(AppiumBy.AccessibilityId("inputB")).SendKeys("5"); Assert.Pass(); } [TearDown] public void TearDown() { if (driver != null) { driver.Quit(); } } } } ``` ```csharp using System; using System.Collections.Generic; using NUnit.Framework; using OpenQA.Selenium; using OpenQA.Selenium.Appium; using OpenQA.Selenium.Appium.iOS; namespace testingbot { [TestFixture] public class TestingBotIOSTest { private IOSDriver driver; [SetUp] public void SetUp() { AppiumOptions appiumOptions = new AppiumOptions(); appiumOptions.PlatformName = "iOS"; appiumOptions.AutomationName = "XCUITest"; appiumOptions.AddAdditionalAppiumOption("appium:deviceName", "iPhone 15"); appiumOptions.AddAdditionalAppiumOption("appium:platformVersion", "18.0"); appiumOptions.AddAdditionalAppiumOption("appium:app", "tb://..."); string key = "TESTINGBOT_KEY"; string secret = "TESTINGBOT_SECRET"; var tbOptions = new Dictionary { ["key"] = key, ["secret"] = secret, ["realDevice"] = true }; appiumOptions.AddAdditionalAppiumOption("tb:options", tbOptions); driver = new IOSDriver( new Uri("https://hub.testingbot.com/wd/hub"), appiumOptions ); } [Test] public void SampleTest() { driver.FindElement(AppiumBy.AccessibilityId("inputA")).SendKeys("10"); driver.FindElement(AppiumBy.AccessibilityId("inputB")).SendKeys("5"); Assert.Pass(); } [TearDown] public void TearDown() { if (driver != null) { driver.Quit(); } } } } ``` ## Uploading Your App Please see our [Upload Mobile App](https://testingbot.com/support/app-automate/help/upload) documentation to find out how to upload your app to TestingBot for testing. ## Specify Devices To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before:** ```csharp IOSDriver driver = new IOSDriver( new Uri("http://localhost:4723/wd/hub"), appiumOptions ); ``` **After:** ```csharp IOSDriver driver = new IOSDriver( new Uri("https://hub.testingbot.com/wd/hub"), appiumOptions ); ``` ### Configuring Capabilities Capabilities are key-value pairs that allows you to customize your tests on TestingBot. Appium provides its [own set of capabilities](https://appium.io/docs/en/2.0/guides/caps/) which you can specify. TestingBot also provide its own [Custom Capabilities](https://testingbot.com/support/app-automate/appium/options), to customize tests ran on the TestingBot platform. You can use the drop-down menus below to see how to configure your tests to run on a specific mobile device: ![OS selected](https://testingbot.com/assets/environments/svg/ios-383468addf160fa18d0e431f529420739d7f7d1206175f682fead627d2e99a52.svg) iOS 18.6 › iPhone 16 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a NUnit C# test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```csharp using System; using System.Collections.Generic; using NUnit.Framework; using OpenQA.Selenium.Appium; using OpenQA.Selenium.Appium.Android; namespace AppiumTest { [TestFixture] public class LocalTunnelTests { [Test] public void TestLocal() { AppiumOptions appiumOptions = new AppiumOptions(); appiumOptions.PlatformName = "Android"; appiumOptions.AutomationName = "UiAutomator2"; appiumOptions.AddAdditionalAppiumOption("appium:deviceName", "Galaxy S23"); appiumOptions.AddAdditionalAppiumOption("appium:platformVersion", "13.0"); appiumOptions.AddAdditionalAppiumOption("appium:app", "tb://..."); var tbOptions = new Dictionary { ["name"] = "Tunnel Test" }; appiumOptions.AddAdditionalAppiumOption("tb:options", tbOptions); // Point to local tunnel instead of hub.testingbot.com AndroidDriver driver = new AndroidDriver( new Uri("http://localhost:4445/wd/hub"), appiumOptions ); // Perform your app test actions here driver.Quit(); } } } ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different device configurations or run different tests all on the same device configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed To see if a test passed or not in our member area, or to send additional meta-data to TestingBot, you can use our API. Please see the example below on how to notify TestingBot about the test success state: ```csharp [TearDown] public void CleanUp() { bool passed = TestContext.CurrentContext.Result.Status == TestStatus.Passed; try { // Logs the result to TestingBot ((IJavaScriptExecutor)driver).ExecuteScript("tb:test-result=" + (passed ? "passed" : "failed")); } finally { // Terminates the remote webdriver session driver.Quit(); } } ``` ## Preparing your App You do not need to install any code or plugin to run a test. Below are some things that are necessary to successfully run a mobile test: **For Android:** - Please supply the URL to your `.apk` or `.aab` file. **Important:** the `.apk` file needs to be a x86 build (x86 ABI) for Android emulators. **For iOS Real Device:** - Please supply the URL to an `.ipa` file. **For iOS Simulator:** - Please supply the URL to a `.zip` file that contains your `.app` - The `.app` needs to be an iOS Simulator build: - Create a Simulator build: ```bash xcodebuild -sdk iphonesimulator -configuration Debug ``` - Zip the `.app` bundle: ```bash zip -r MyApp.zip MyApp.app ``` ## Additional Options Appium provides [a lot of options](https://appium.io/docs/en/2.0/guides/caps/) to configure your test. Some important options that might help: **For Android:** - **appActivity** and **appPackage** : by default, Appium will try to extract the main Activity from your apk. If this fails, you can supply your own with these options. - **chromeOptions** : additional chromedriver options you can supply. - **otherApps** : a JSON array of other apps that need to be installed, in URL format. **For Android & iOS:** - **locale** : the language of the simulator/device (ex: `fr_CA`) This sets the locale on the entire device/simulator, except on physical iOS devices. For real iOS devices, we will pass `-AppleLocale` as argument to the app. - **newCommandTimeout** : How long (in seconds) Appium will wait for a new command from the client before assuming the client quit and ending the session ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/appium/flutter # Automated Flutter App Testing TestingBot supports automated testing of Flutter apps via [Appium's Flutter Driver](https://github.com/appium-userland/appium-flutter-driver). To get started, please make sure to follow these steps: 1. Compile your Flutter app in `debug` or `profile` mode. Appium's Flutter driver does not currently support apps built in `release` mode. 2. Make sure your app's `pubspec.yaml` file contains: 3. Import the new `dev_dependencies` that you have added in the previous step: 4. Next, import the `flutter_driver_extension` library in your `main.dart` file: 5. Your `main.dart` file should contain `enableFlutterDriverExtension()` before `runApp`. 6. The `automationName` in your desired capabilities should be set to `Flutter`. ## Preparing your App You can build an `.apk` file for Android, `.ipa` file for iOS physical devices or `.app` bundle for iOS Simulator testing. This file needs to be [uploaded](https://testingbot.com#upload) to TestingBot for automated or manual testing. Please see the table below on how to build your app: | OS | Build Mode | Command | | --- | --- | --- | | Android | debug | ```bash flutter build apk --debug ``` | | | profile | ```bash flutter build apk --profile ``` | | iOS | debug | ```bash flutter build ios --debug ``` | | | simulator | ```bash flutter build ios --simulator ``` | | | profile | ```bash flutter build ios --profile ``` | - For iOS 14 and higher, please build the app in `profile` mode only. - Release mode is not supported by Appium's Flutter Driver. - For iOS real device testing, please use `flutter build ipa`, then do: ```bash mkdir Payload mv Runner.app Payload/ zip --symlinks -qr test-app.ipa Payload ``` ## Uploading Your App Please see our [Upload Mobile App](https://testingbot.com/support/app-automate/help/upload) documentation to find out how to upload your app to TestingBot for testing. ## Example TestingBot has created a [flutter-demo-app](https://github.com/testingbot/flutter-demo-app) which you can use to run your first Flutter Automated test with TestingBot. Please see the example below where we run an automated Flutter test on TestingBot. ```javascript const { remote } = require('webdriverio'); const assert = require('assert'); const { byValueKey } = require('appium-flutter-finder'); const capabilities = { 'platformName': 'Android', 'appium:deviceName': 'Pixel 8', 'appium:platformVersion': '14.0', 'appium:automationName': 'Flutter', 'appium:app': 'https://github.com/testingbot/flutter-demo-app/releases/download/v1.0.0/demo.apk', 'tb:options': { 'realDevice': true } }; (async () => { const driver = await remote({ hostname: 'hub.testingbot.com', port: 443, protocol: 'https', path: '/wd/hub', user: 'api_key', key: 'api_secret', capabilities }); const counterTextFinder = byValueKey('counter'); const buttonFinder = byValueKey('incrementButton'); // appium-flutter-finder returns a finder you pass to standard WebdriverIO element commands assert.strictEqual(await driver.getElementText(counterTextFinder), '0'); await driver.elementClick(buttonFinder); await driver.elementClick(buttonFinder); assert.strictEqual(await driver.getElementText(counterTextFinder), '2'); await driver.deleteSession(); })(); ``` This example test will open the [flutter-demo-app](https://github.com/testingbot/flutter-demo-app), find both the counter number and increment button by `valueKey`. It will then verify the initial number, click the button twice and verify if the number is correct. You can use the following demo applications to try out automated Flutter testing: - For iOS simulator testing: ```bash https://github.com/testingbot/flutter-demo-app/releases/download/v1.0.0/demo-ios-simulator.zip ``` - For iOS physical device testing: ```bash https://github.com/testingbot/flutter-demo-app/releases/download/v1.0.0/demo.ipa ``` - For Android testing: ```bash https://github.com/testingbot/flutter-demo-app/releases/download/v1.0.0/demo.apk ``` ## Specify Devices To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote devices. If the test was running on your local machine or network, you can simply change your existing test like this: To let TestingBot know on which device you want to run your test on, you need to specify the `appium:deviceName`, `appium:platformVersion`, `platformName` and other optional options in the capabilities field. ![OS selected](https://testingbot.com/assets/environments/svg/ios-383468addf160fa18d0e431f529420739d7f7d1206175f682fead627d2e99a52.svg) iOS 18.6 › iPhone 16 Loading environments... Please wait while we load the available browsers and platforms. ## Additional Options Appium provides [multiple Flutter capabilities](https://github.com/appium/appium-flutter-driver#capabilities) to configure your test. Additionally, you can use [Flutter Finders](https://github.com/appium/appium-flutter-driver#Finders) and [Flutter Commands](https://github.com/appium/appium-flutter-driver#commands). See our [Appium Flutter Driver](https://testingbot.com/resources/articles/appium-flutter-app-testing) article for background on what Flutter compiles to and how to build an app that the driver can inspect. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/appium/java ### Java Examples: - [TestNG](https://testingbot.com/support/app-automate/appium/java/testng) # JUnit Automated App Testing See our [JUnit example repository](https://github.com/testingbot/java-junit-app-example) for a simple example on how to run JUnit mobile app tests. ## Installation First, make sure you install the necessary dependencies to run a test. For this example, we'll be using [JUnit 5](https://junit.org/junit5/) and the [Appium Java Client](https://github.com/appium/java-client). Add the following dependencies to your `pom.xml`: ```xml org.junit.jupiter junit-jupiter 5.10.2 test io.appium java-client 10.1.1 test ``` **Note:** Appium Java Client 10.x requires Java 11 or higher. ## JUnit Example The example below shows a complete JUnit 5 test that connects to the TestingBot hub and runs a test on a real device. In this test, we'll test a sample calculator app by entering values into two input fields and verifying the sum. [Android](https://testingbot.com#) [iOS](https://testingbot.com#) ```java import io.appium.java_client.AppiumBy; import io.appium.java_client.android.AndroidDriver; import io.appium.java_client.android.options.UiAutomator2Options; import org.junit.jupiter.api.*; import org.openqa.selenium.WebElement; import org.openqa.selenium.support.ui.ExpectedConditions; import org.openqa.selenium.support.ui.WebDriverWait; import java.net.MalformedURLException; import java.net.URL; import java.time.Duration; import static org.junit.jupiter.api.Assertions.assertEquals; public class AndroidAppTest { private AndroidDriver driver; @BeforeEach public void setUp() throws MalformedURLException { // Configure capabilities using UiAutomator2Options UiAutomator2Options options = new UiAutomator2Options() .setPlatformName("Android") .setPlatformVersion("16.0") .setDeviceName("Pixel 9") .setApp("tb://") .setAutomationName("UiAutomator2"); // Add TestingBot-specific options options.setCapability("tb:options", java.util.Map.of( "key", "YOUR_TB_KEY", "secret", "YOUR_TB_SECRET", "name", "Android Calculator Test", "build", "JUnit Build 1" )); // Connect to TestingBot hub driver = new AndroidDriver( new URL("https://hub.testingbot.com/wd/hub"), options ); } @Test public void testCalculatorSum() { WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(30)); // Enter first number WebElement inputA = wait.until( ExpectedConditions.elementToBeClickable(AppiumBy.accessibilityId("inputA")) ); inputA.sendKeys("10"); // Enter second number WebElement inputB = wait.until( ExpectedConditions.elementToBeClickable(AppiumBy.accessibilityId("inputB")) ); inputB.sendKeys("5"); // Verify the sum WebElement sum = wait.until( ExpectedConditions.elementToBeClickable(AppiumBy.accessibilityId("sum")) ); assertEquals("15", sum.getText()); } @AfterEach public void tearDown() { if (driver != null) { driver.quit(); } } } ``` ```java import io.appium.java_client.AppiumBy; import io.appium.java_client.ios.IOSDriver; import io.appium.java_client.ios.options.XCUITestOptions; import org.junit.jupiter.api.*; import org.openqa.selenium.WebElement; import org.openqa.selenium.support.ui.ExpectedConditions; import org.openqa.selenium.support.ui.WebDriverWait; import java.net.MalformedURLException; import java.net.URL; import java.time.Duration; import static org.junit.jupiter.api.Assertions.assertEquals; public class IOSAppTest { private IOSDriver driver; @BeforeEach public void setUp() throws MalformedURLException { // Configure capabilities using XCUITestOptions XCUITestOptions options = new XCUITestOptions() .setPlatformName("iOS") .setPlatformVersion("26.0") .setDeviceName("iPhone 17") .setApp("tb://") .setAutomationName("XCUITest"); // Add TestingBot-specific options options.setCapability("tb:options", java.util.Map.of( "key", "YOUR_TB_KEY", "secret", "YOUR_TB_SECRET", "name", "iOS Calculator Test", "build", "JUnit Build 1" )); // Connect to TestingBot hub driver = new IOSDriver( new URL("https://hub.testingbot.com/wd/hub"), options ); } @Test public void testCalculatorSum() { WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(30)); // Enter first number WebElement inputA = wait.until( ExpectedConditions.elementToBeClickable(AppiumBy.accessibilityId("inputA")) ); inputA.sendKeys("10"); // Enter second number WebElement inputB = wait.until( ExpectedConditions.elementToBeClickable(AppiumBy.accessibilityId("inputB")) ); inputB.sendKeys("5"); // Verify the sum WebElement sum = wait.until( ExpectedConditions.elementToBeClickable(AppiumBy.accessibilityId("sum")) ); assertEquals("15", sum.getText()); } @AfterEach public void tearDown() { if (driver != null) { driver.quit(); } } } ``` Always call `driver.quit()` in your teardown method to properly end the session. Otherwise, the test will continue running until it times out. ## Uploading Your App Please see our [Upload Mobile App](https://testingbot.com/support/app-automate/help/upload) documentation to find out how to upload your app to TestingBot for testing. ## Specify Browsers & Devices To run your tests on TestingBot, you need to configure the device and platform you want to test on. The Appium Java client provides type-safe Options classes for this purpose. ### Using Options Classes (Recommended) For Android testing, use `UiAutomator2Options`. For iOS testing, use `XCUITestOptions`: [Android](https://testingbot.com#) [iOS](https://testingbot.com#) ```java UiAutomator2Options options = new UiAutomator2Options() .setPlatformName("Android") .setPlatformVersion("16.0") .setDeviceName("Pixel 9") .setApp("tb://") .setAutomationName("UiAutomator2"); // Add TestingBot credentials options.setCapability("tb:options", Map.of( "key", "YOUR_TB_KEY", "secret", "YOUR_TB_SECRET", "name", "My Test", "build", "Build 1" )); AndroidDriver driver = new AndroidDriver( new URL("https://hub.testingbot.com/wd/hub"), options ); ``` ```java XCUITestOptions options = new XCUITestOptions() .setPlatformName("iOS") .setPlatformVersion("26.0") .setDeviceName("iPhone 17") .setApp("tb://") .setAutomationName("XCUITest"); // Add TestingBot credentials options.setCapability("tb:options", Map.of( "key", "YOUR_TB_KEY", "secret", "YOUR_TB_SECRET", "name", "My Test", "build", "Build 1" )); IOSDriver driver = new IOSDriver( new URL("https://hub.testingbot.com/wd/hub"), options ); ``` ### Using BaseOptions (Generic) For more flexibility or when working with multiple platforms, you can use `BaseOptions`: ```java import io.appium.java_client.AppiumDriver; import io.appium.java_client.remote.options.BaseOptions; import org.openqa.selenium.MutableCapabilities; import java.net.URL; import java.util.Map; BaseOptions options = new BaseOptions<>() .setPlatformName("Android") .amend("appium:automationName", "UiAutomator2") .amend("appium:deviceName", "Pixel 9") .amend("appium:platformVersion", "16.0") .amend("appium:app", "tb://"); options.setCapability("tb:options", Map.of( "key", "YOUR_TB_KEY", "secret", "YOUR_TB_SECRET" )); AppiumDriver driver = new AppiumDriver( new URL("https://hub.testingbot.com/wd/hub"), options ); ``` ### Required Capabilities To specify which device you want to test on, you need to provide: - `platformName` - The mobile OS (Android or iOS) - `appium:platformVersion` - The OS version (e.g., "17.0", "26.0") - `appium:deviceName` - The device name (e.g., "Pixel 9", "iPhone 17") - `appium:app` - The app to test (URL or tb:// reference) - `appium:automationName` - The automation driver ("UiAutomator2" for Android, "XCUITest" for iOS) Use the device picker below to generate the configuration for your target device: ![OS selected](https://testingbot.com/assets/environments/svg/ios-383468addf160fa18d0e431f529420739d7f7d1206175f682fead627d2e99a52.svg) iOS 18.6 › iPhone 16 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a JUnit test with our Tunnel: **1.** [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` **2.** Update your test to connect through the tunnel instead of directly to the TestingBot hub. Change the URL from `hub.testingbot.com` to `localhost:4445`: ```java // Connect through the tunnel instead of directly to TestingBot AndroidDriver driver = new AndroidDriver( new URL("http://localhost:4445/wd/hub"), options ); ``` Your test traffic will now go securely through the tunnel, allowing you to test apps that access internal resources. ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. Please see our [Parallel JUnit documentation](https://testingbot.com/support/web-automate/selenium/java/parallel-junit) for parallel testing. ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed TestingBot cannot automatically determine if your test passed or failed - that depends on your test assertions. To report test results to TestingBot, use our [REST API](https://testingbot.com/support/api). First, add the [TestingBot Java client](https://github.com/testingbot/testingbot-java) to your dependencies: ```xml com.testingbot testingbotrest 1.0.10 ``` Then use it in your test to report the result: ```java import com.testingbot.comingbotrest.TestingbotREST; import java.util.Map; @AfterEach public void tearDown() { if (driver != null) { // Report test result to TestingBot TestingbotREST api = new TestingbotREST( "YOUR_TB_KEY", "YOUR_TB_SECRET" ); api.updateTest( driver.getSessionId().toString(), Map.of( "success", "1", // "1" for passed, "0" for failed "name", "Calculator Test" ) ); driver.quit(); } } ``` For JUnit 5, you can use a `TestWatcher` extension to automatically report results: ```java import org.junit.jupiter.api.extension.*; public class TestingBotWatcher implements TestWatcher { private final AndroidDriver driver; private final TestingbotREST api; public TestingBotWatcher(AndroidDriver driver, String key, String secret) { this.driver = driver; this.api = new TestingbotREST(key, secret); } @Override public void testSuccessful(ExtensionContext context) { updateTest("1", context.getDisplayName()); } @Override public void testFailed(ExtensionContext context, Throwable cause) { updateTest("0", context.getDisplayName()); } private void updateTest(String success, String name) { api.updateTest(driver.getSessionId().toString(), Map.of( "success", success, "name", name )); } } ``` See our [Appium Page Object Model & Page Factory](https://testingbot.com/resources/articles/page-object-model-appium) article for keeping a Java suite readable as it grows, including `@FindBy` locators and when to cache lookups. ## Preparing your App You do not need to install any code or plugin to run a test. Below are some things that are necessary to successfully run a mobile test: **For Android:** - Please supply the URL to your `.apk` or `.aab` file. **Important:** the `.apk` file needs to be a x86 build (x86 ABI) for Android emulators. **For iOS Real Device:** - Please supply the URL to an `.ipa` file. **For iOS Simulator:** - Please supply the URL to a `.zip` file that contains your `.app` - The `.app` needs to be an iOS Simulator build: - Create a Simulator build: ```bash xcodebuild -sdk iphonesimulator -configuration Debug ``` - Zip the `.app` bundle: ```bash zip -r MyApp.zip MyApp.app ``` ## Additional Options Appium provides [a lot of options](https://appium.io/docs/en/2.0/guides/caps/) to configure your test. Some important options that might help: **For Android:** - **appActivity** and **appPackage** : by default, Appium will try to extract the main Activity from your apk. If this fails, you can supply your own with these options. - **chromeOptions** : additional chromedriver options you can supply. - **otherApps** : a JSON array of other apps that need to be installed, in URL format. **For Android & iOS:** - **locale** : the language of the simulator/device (ex: `fr_CA`) This sets the locale on the entire device/simulator, except on physical iOS devices. For real iOS devices, we will pass `-AppleLocale` as argument to the app. - **newCommandTimeout** : How long (in seconds) Appium will wait for a new command from the client before assuming the client quit and ending the session ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/appium/java/testng # TestNG Automated App Testing See our [TestNG example repository](https://github.com/testingbot/java-testng-app-example) for a simple example on how to run TestNG mobile app tests. ## Installation First, make sure you install the necessary dependencies to run a test. For this example, we'll be using [TestNG](https://testng.org/) and the [Appium Java Client](https://github.com/appium/java-client). Add the following dependencies to your `pom.xml`: ```xml org.testng testng 7.11.0 test io.appium java-client 10.1.1 test ``` **Note:** Appium Java Client 10.x requires Java 11 or higher. ## TestNG Example The example below shows a complete TestNG test that connects to the TestingBot hub and runs a test on a real device. In this test, we'll test a sample calculator app by entering values into two input fields and verifying the sum. [Android](https://testingbot.com#) [iOS](https://testingbot.com#) ```java import io.appium.java_client.AppiumBy; import io.appium.java_client.android.AndroidDriver; import io.appium.java_client.android.options.UiAutomator2Options; import org.openqa.selenium.WebElement; import org.openqa.selenium.support.ui.ExpectedConditions; import org.openqa.selenium.support.ui.WebDriverWait; import org.testng.Assert; import org.testng.annotations.*; import java.net.MalformedURLException; import java.net.URL; import java.time.Duration; import java.util.Map; public class AndroidAppTest { private AndroidDriver driver; @BeforeMethod public void setUp() throws MalformedURLException { // Configure capabilities using UiAutomator2Options UiAutomator2Options options = new UiAutomator2Options() .setPlatformName("Android") .setPlatformVersion("16.0") .setDeviceName("Pixel 9") .setApp("tb://") .setAutomationName("UiAutomator2"); // Add TestingBot-specific options options.setCapability("tb:options", Map.of( "key", "YOUR_TB_KEY", "secret", "YOUR_TB_SECRET", "name", "Android Calculator Test", "build", "TestNG Build 1" )); // Connect to TestingBot hub driver = new AndroidDriver( new URL("https://hub.testingbot.com/wd/hub"), options ); } @Test public void testCalculatorSum() { WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(30)); // Enter first number WebElement inputA = wait.until( ExpectedConditions.elementToBeClickable(AppiumBy.accessibilityId("inputA")) ); inputA.sendKeys("10"); // Enter second number WebElement inputB = wait.until( ExpectedConditions.elementToBeClickable(AppiumBy.accessibilityId("inputB")) ); inputB.sendKeys("5"); // Verify the sum WebElement sum = wait.until( ExpectedConditions.elementToBeClickable(AppiumBy.accessibilityId("sum")) ); Assert.assertEquals(sum.getText(), "15"); } @AfterMethod public void tearDown() { if (driver != null) { driver.quit(); } } } ``` ```java import io.appium.java_client.AppiumBy; import io.appium.java_client.ios.IOSDriver; import io.appium.java_client.ios.options.XCUITestOptions; import org.openqa.selenium.WebElement; import org.openqa.selenium.support.ui.ExpectedConditions; import org.openqa.selenium.support.ui.WebDriverWait; import org.testng.Assert; import org.testng.annotations.*; import java.net.MalformedURLException; import java.net.URL; import java.time.Duration; import java.util.Map; public class IOSAppTest { private IOSDriver driver; @BeforeMethod public void setUp() throws MalformedURLException { // Configure capabilities using XCUITestOptions XCUITestOptions options = new XCUITestOptions() .setPlatformName("iOS") .setPlatformVersion("26.0") .setDeviceName("iPhone 17") .setApp("tb://") .setAutomationName("XCUITest"); // Add TestingBot-specific options options.setCapability("tb:options", Map.of( "key", "YOUR_TB_KEY", "secret", "YOUR_TB_SECRET", "name", "iOS Calculator Test", "build", "TestNG Build 1" )); // Connect to TestingBot hub driver = new IOSDriver( new URL("https://hub.testingbot.com/wd/hub"), options ); } @Test public void testCalculatorSum() { WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(30)); // Enter first number WebElement inputA = wait.until( ExpectedConditions.elementToBeClickable(AppiumBy.accessibilityId("inputA")) ); inputA.sendKeys("10"); // Enter second number WebElement inputB = wait.until( ExpectedConditions.elementToBeClickable(AppiumBy.accessibilityId("inputB")) ); inputB.sendKeys("5"); // Verify the sum WebElement sum = wait.until( ExpectedConditions.elementToBeClickable(AppiumBy.accessibilityId("sum")) ); Assert.assertEquals(sum.getText(), "15"); } @AfterMethod public void tearDown() { if (driver != null) { driver.quit(); } } } ``` Always call `driver.quit()` in your `@AfterMethod` to properly end the session. Otherwise, the test will continue running until it times out. ## Reusable Base Test Class For cleaner test organization, you can create a reusable base class that handles driver setup and teardown. Tests can then extend this base class and focus only on test logic. ### Base Class ```java import io.appium.java_client.android.AndroidDriver; import io.appium.java_client.android.options.UiAutomator2Options; import org.testng.annotations.*; import java.net.MalformedURLException; import java.net.URL; import java.util.Map; public class TestingBotBaseTest { protected AndroidDriver driver; @BeforeMethod(alwaysRun = true) @Parameters({"platformVersion", "deviceName", "app"}) public void setUp( @Optional("16.0") String platformVersion, @Optional("Pixel 9") String deviceName, @Optional("tb://") String app ) throws MalformedURLException { // Get credentials from environment or use defaults String key = System.getenv("TB_KEY"); String secret = System.getenv("TB_SECRET"); if (key == null || secret == null) { throw new RuntimeException("TB_KEY and TB_SECRET environment variables must be set"); } // Configure capabilities UiAutomator2Options options = new UiAutomator2Options() .setPlatformName("Android") .setPlatformVersion(platformVersion) .setDeviceName(deviceName) .setApp(app) .setAutomationName("UiAutomator2"); options.setCapability("tb:options", Map.of( "key", key, "secret", secret, "name", this.getClass().getSimpleName() )); driver = new AndroidDriver( new URL("https://hub.testingbot.com/wd/hub"), options ); } @AfterMethod(alwaysRun = true) public void tearDown() { if (driver != null) { driver.quit(); } } } ``` ### Test Class Using the Base ```java import io.appium.java_client.AppiumBy; import org.openqa.selenium.WebElement; import org.openqa.selenium.support.ui.ExpectedConditions; import org.openqa.selenium.support.ui.WebDriverWait; import org.testng.Assert; import org.testng.annotations.Test; import java.time.Duration; public class CalculatorTest extends TestingBotBaseTest { @Test public void testAddition() { WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(30)); WebElement inputA = wait.until( ExpectedConditions.elementToBeClickable(AppiumBy.accessibilityId("inputA")) ); inputA.sendKeys("10"); WebElement inputB = wait.until( ExpectedConditions.elementToBeClickable(AppiumBy.accessibilityId("inputB")) ); inputB.sendKeys("5"); WebElement sum = wait.until( ExpectedConditions.elementToBeClickable(AppiumBy.accessibilityId("sum")) ); Assert.assertEquals(sum.getText(), "15"); } } ``` ### TestNG XML Configuration Use a `testng.xml` file to pass parameters and run tests on multiple devices: ```xml ``` ## Uploading Your App Please see our [Upload Mobile App](https://testingbot.com/support/app-automate/help/upload) documentation to find out how to upload your app to TestingBot for testing. ## Specify Browsers & Devices To run your tests on TestingBot, you need to configure the device and platform you want to test on. The Appium Java client provides type-safe Options classes for this purpose. ### Using Options Classes (Recommended) For Android testing, use `UiAutomator2Options`. For iOS testing, use `XCUITestOptions`: [Android](https://testingbot.com#) [iOS](https://testingbot.com#) ```java UiAutomator2Options options = new UiAutomator2Options() .setPlatformName("Android") .setPlatformVersion("16.0") .setDeviceName("Pixel 9") .setApp("tb://") .setAutomationName("UiAutomator2"); // Add TestingBot credentials options.setCapability("tb:options", Map.of( "key", "YOUR_TB_KEY", "secret", "YOUR_TB_SECRET", "name", "My Test", "build", "Build 1" )); AndroidDriver driver = new AndroidDriver( new URL("https://hub.testingbot.com/wd/hub"), options ); ``` ```java XCUITestOptions options = new XCUITestOptions() .setPlatformName("iOS") .setPlatformVersion("26.0") .setDeviceName("iPhone 17") .setApp("tb://") .setAutomationName("XCUITest"); // Add TestingBot credentials options.setCapability("tb:options", Map.of( "key", "YOUR_TB_KEY", "secret", "YOUR_TB_SECRET", "name", "My Test", "build", "Build 1" )); IOSDriver driver = new IOSDriver( new URL("https://hub.testingbot.com/wd/hub"), options ); ``` ### Using BaseOptions (Generic) For more flexibility or when working with multiple platforms, you can use `BaseOptions`: ```java import io.appium.java_client.remote.options.BaseOptions; BaseOptions options = new BaseOptions() .setPlatformName("Android") .setAutomationName("UiAutomator2") .amend("appium:deviceName", "Pixel 9") .amend("appium:platformVersion", "16.0") .amend("appium:app", "tb://"); options.setCapability("tb:options", Map.of( "key", "YOUR_TB_KEY", "secret", "YOUR_TB_SECRET" )); AppiumDriver driver = new AppiumDriver( new URL("https://hub.testingbot.com/wd/hub"), options ); ``` ### Required Capabilities To specify which device you want to test on, you need to provide: - `platformName` - The mobile OS (Android or iOS) - `appium:platformVersion` - The OS version (e.g., "16.0", "26.0") - `appium:deviceName` - The device name (e.g., "Pixel 9", "iPhone 17") - `appium:app` - The app to test (URL or tb:// reference) - `appium:automationName` - The automation driver ("UiAutomator2" for Android, "XCUITest" for iOS) Use the device picker below to generate the configuration for your target device: ![OS selected](https://testingbot.com/assets/environments/svg/ios-383468addf160fa18d0e431f529420739d7f7d1206175f682fead627d2e99a52.svg) iOS 18.6 › iPhone 16 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a TestNG test with our Tunnel: **1.** [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` **2.** Update your test to connect through the tunnel instead of directly to the TestingBot hub. Change the URL from `hub.testingbot.com` to `localhost:4445`: ```java // Connect through the tunnel instead of directly to TestingBot AndroidDriver driver = new AndroidDriver( new URL("http://localhost:4445/wd/hub"), options ); ``` Your test traffic will now go securely through the tunnel, allowing you to test apps that access internal resources. ## Mark tests as passed/failed TestingBot cannot automatically determine if your test passed or failed - that depends on your test assertions. To report test results to TestingBot, use our [REST API](https://testingbot.com/support/api). First, add the [TestingBot Java client](https://github.com/testingbot/testingbot-java) to your dependencies: ```xml com.testingbot testingbotrest 1.0.10 ``` Then use it in your test to report the result: ```java import com.testingbot.comingbotrest.TestingbotREST; import java.util.Map; @AfterMethod public void tearDown() { if (driver != null) { // Report test result to TestingBot TestingbotREST api = new TestingbotREST( "YOUR_TB_KEY", "YOUR_TB_SECRET" ); api.updateTest( driver.getSessionId().toString(), Map.of( "success", "1", // "1" for passed, "0" for failed "name", "Calculator Test" ) ); driver.quit(); } } ``` For automatic result reporting, you can use a TestNG listener: ```java import org.testng.ITestListener; import org.testng.ITestResult; import com.testingbot.comingbotrest.TestingbotREST; import java.util.Map; public class TestingBotListener implements ITestListener { private TestingbotREST api; public TestingBotListener() { this.api = new TestingbotREST( System.getenv("TB_KEY"), System.getenv("TB_SECRET") ); } @Override public void onTestSuccess(ITestResult result) { updateTestStatus(result, "1"); } @Override public void onTestFailure(ITestResult result) { updateTestStatus(result, "0"); } private void updateTestStatus(ITestResult result, String success) { Object testInstance = result.getInstance(); if (testInstance instanceof TestingBotBaseTest) { AndroidDriver driver = ((TestingBotBaseTest) testInstance).driver; if (driver != null) { api.updateTest( driver.getSessionId().toString(), Map.of( "success", success, "name", result.getMethod().getMethodName() ) ); } } } } ``` Register the listener in your `testng.xml`: ```xml ``` ## Preparing your App You do not need to install any code or plugin to run a test. Below are some things that are necessary to successfully run a mobile test: **For Android:** - Please supply the URL to your `.apk` or `.aab` file. **Important:** the `.apk` file needs to be a x86 build (x86 ABI) for Android emulators. **For iOS Real Device:** - Please supply the URL to an `.ipa` file. **For iOS Simulator:** - Please supply the URL to a `.zip` file that contains your `.app` - The `.app` needs to be an iOS Simulator build: - Create a Simulator build: ```bash xcodebuild -sdk iphonesimulator -configuration Debug ``` - Zip the `.app` bundle: ```bash zip -r MyApp.zip MyApp.app ``` ## Additional Options Appium provides [a lot of options](https://appium.io/docs/en/2.0/guides/caps/) to configure your test. Some important options that might help: **For Android:** - **appActivity** and **appPackage** : by default, Appium will try to extract the main Activity from your apk. If this fails, you can supply your own with these options. - **chromeOptions** : additional chromedriver options you can supply. - **otherApps** : a JSON array of other apps that need to be installed, in URL format. **For Android & iOS:** - **locale** : the language of the simulator/device (ex: `fr_CA`) This sets the locale on the entire device/simulator, except on physical iOS devices. For real iOS devices, we will pass `-AppleLocale` as argument to the app. - **newCommandTimeout** : How long (in seconds) Appium will wait for a new command from the client before assuming the client quit and ending the session ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/appium/nodejs ### NodeJS Examples: - [WebDriverIO](https://testingbot.com/support/app-automate/appium/nodejs/webdriverio) - [CodeceptJS](https://testingbot.com/support/app-automate/appium/nodejs/codeceptjs) # NodeJS Automated App Testing First, make sure you have installed NodeJS. If you haven't, you can easily install it by downloading NodeJS from the [NodeJS website](https://nodejs.org/en/download/) ## Installation With TestingBot you can easily run your automated tests with any NodeJS test framework, here's a simple example with **WebdriverIO** : ```bash npm install webdriverio ``` ## Real Device Testing [Android](https://testingbot.com#) [iOS](https://testingbot.com#) This will download and install a sample Android app we built. It will add two numbers to a textinput and display the sum of the two numbers. ```javascript const { remote } = require('webdriverio'); const capabilities = { hostname: 'hub.testingbot.com', port: 443, protocol: 'https', path: '/wd/hub', user: "api_key", key: "api_secret", capabilities: { 'appium:deviceName': 'Pixel 9', 'platformName': 'Android', 'appium:platformVersion': '16.0', 'appium:automationName': 'UiAutomator2', 'appium:app': 'https://testingbot.com/appium/sample.apk', 'tb:options': { 'name': 'My First App Test', 'realDevice': true } } }; (async () => { const driver = await remote(capabilities); const inputA = await driver.$('~inputA'); await inputA.waitForDisplayed({ timeout: 30000 }); await inputA.setValue('10'); const inputB = await driver.$('~inputB'); await inputB.waitForDisplayed({ timeout: 30000 }); await inputB.setValue('5'); const sum = await driver.$('~sum'); await sum.waitForDisplayed({ timeout: 30000 }); const value = await sum.getText(); console.log(value === '15' ? 'Test passed!' : 'Test failed!'); await driver.deleteSession(); })(); ``` Make sure to always end your session (`await driver.deleteSession()`), otherwise it will continue running, leading to a timeout. This will download and install a sample iOS app we built. It will add two numbers to a textinput and display the sum of the two numbers. ```javascript const { remote } = require('webdriverio'); const capabilities = { hostname: 'hub.testingbot.com', port: 443, protocol: 'https', path: '/wd/hub', user: "api_key", key: "api_secret", capabilities: { 'appium:deviceName': 'iPhone 17', 'platformName': 'iOS', 'appium:platformVersion': '26.0', 'appium:automationName': 'XCUITest', 'appium:app': 'https://testingbot.com/appium/sample.ipa', 'tb:options': { 'name': 'My First App Test', 'realDevice': true } } }; (async () => { const driver = await remote(capabilities); const inputA = await driver.$('~inputA'); await inputA.waitForDisplayed({ timeout: 30000 }); await inputA.setValue('10'); const inputB = await driver.$('~inputB'); await inputB.waitForDisplayed({ timeout: 30000 }); await inputB.setValue('5'); await driver.deleteSession(); })(); ``` Make sure to always end your session (`await driver.deleteSession()`), otherwise it will continue running, leading to a timeout. ## Simulator/Emulator Testing [Android](https://testingbot.com#) [iOS](https://testingbot.com#) This will download and install a sample Android app we built. It will add two numbers to a textinput and display the sum of the two numbers. ```javascript const { remote } = require('webdriverio'); const capabilities = { hostname: 'hub.testingbot.com', port: 443, protocol: 'https', path: '/wd/hub', user: "api_key", key: "api_secret", capabilities: { 'appium:deviceName': 'Pixel 9', 'platformName': 'Android', 'appium:platformVersion': '16.0', 'appium:automationName': 'UiAutomator2', 'appium:app': 'https://testingbot.com/appium/sample.apk', 'tb:options': { 'name': 'My First App Test' } } }; (async () => { const driver = await remote(capabilities); const inputA = await driver.$('~inputA'); await inputA.waitForDisplayed({ timeout: 30000 }); await inputA.setValue('10'); const inputB = await driver.$('~inputB'); await inputB.waitForDisplayed({ timeout: 30000 }); await inputB.setValue('5'); const sum = await driver.$('~sum'); await sum.waitForDisplayed({ timeout: 30000 }); const value = await sum.getText(); console.log(value === '15' ? 'Test passed!' : 'Test failed!'); await driver.deleteSession(); })(); ``` Make sure to always end your session (`await driver.deleteSession()`), otherwise it will continue running, leading to a timeout. This will download and install a sample iOS app we built. It will add two numbers to a textinput and display the sum of the two numbers. ```javascript const { remote } = require('webdriverio'); const capabilities = { hostname: 'hub.testingbot.com', port: 443, protocol: 'https', path: '/wd/hub', user: "api_key", key: "api_secret", capabilities: { 'appium:deviceName': 'iPhone 17', 'platformName': 'iOS', 'appium:platformVersion': '26.0', 'appium:automationName': 'XCUITest', 'appium:app': 'https://testingbot.com/appium/sample.zip', 'tb:options': { 'name': 'My First App Test' } } }; (async () => { const driver = await remote(capabilities); const inputA = await driver.$('~inputA'); await inputA.waitForDisplayed({ timeout: 30000 }); await inputA.setValue('10'); const inputB = await driver.$('~inputB'); await inputB.waitForDisplayed({ timeout: 30000 }); await inputB.setValue('5'); await driver.deleteSession(); })(); ``` Make sure to always end your session (`await driver.deleteSession()`), otherwise it will continue running, leading to a timeout. ## Uploading Your App Please see our [Upload Mobile App](https://testingbot.com/support/app-automate/help/upload) documentation to find out how to upload your app to TestingBot for testing. ## Specify Devices To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before:** ```javascript const capabilities = { hostname: 'localhost', port: 4723, ... }; ``` **After:** ```javascript const capabilities = { hostname: 'hub.testingbot.com', port: 443, protocol: 'https', path: '/wd/hub', user: 'api_key', key: 'api_secret', capabilities: { 'appium:app': 'tb://...' } }; ``` To let TestingBot know on which device you want to run your test on, you need to specify the `appium:deviceName`, `appium:platformVersion`, `platformName` and other optional options in the capabilities field. To see how to do this, please select a combination of device type and device name in the drop-down menus below. ![OS selected](https://testingbot.com/assets/environments/svg/ios-383468addf160fa18d0e431f529420739d7f7d1206175f682fead627d2e99a52.svg) iOS 18.6 › iPhone 16 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a NodeJS test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```javascript const { remote } = require('webdriverio'); const capabilities = { hostname: 'localhost', port: 4445, protocol: 'http', path: '/wd/hub', user: "api_key", key: "api_secret", capabilities: { 'appium:deviceName': 'iPhone 17', 'platformName': 'iOS', 'appium:platformVersion': '26.0', 'appium:automationName': 'XCUITest', 'appium:app': 'https://testingbot.com/appium/sample.zip', 'tb:options': { 'name': 'My First App Test' } } }; (async () => { const driver = await remote(capabilities); const inputA = await driver.$('~inputA'); await inputA.waitForDisplayed({ timeout: 30000 }); await inputA.setValue('10'); const inputB = await driver.$('~inputB'); await inputB.waitForDisplayed({ timeout: 30000 }); await inputB.setValue('5'); await driver.deleteSession(); })(); ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. To run tests in parallel, we recommend using a test runner like [WebDriverIO](https://testingbot.com/support/app-automate/appium/nodejs/webdriverio) to run your tests simultaneously. ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. Install our NPM package to interact with the TestingBot API: ```bash npm install testingbot-api ``` After your test completed, you can send back the test success state and other meta-data: ```javascript const TestingBot = require('testingbot-api'); const tb = new TestingBot({ api_key: "api_key", api_secret: "api_secret" }); // Obtain the session id from your WebDriver session // const sessionId = driver.sessionId; // (WebdriverIO) tb.updateTest({ name: 'Example Test', success: true }, sessionId, (err) => { if (err) console.error(err); }); ``` ## Preparing your App You do not need to install any code or plugin to run a test. Below are some things that are necessary to successfully run a mobile test: **For Android:** - Please supply the URL to your `.apk` or `.aab` file. **Important:** the `.apk` file needs to be a x86 build (x86 ABI) for Android emulators. **For iOS Real Device:** - Please supply the URL to an `.ipa` file. **For iOS Simulator:** - Please supply the URL to a `.zip` file that contains your `.app` - The `.app` needs to be an iOS Simulator build: - Create a Simulator build: ```bash xcodebuild -sdk iphonesimulator -configuration Debug ``` - Zip the `.app` bundle: ```bash zip -r MyApp.zip MyApp.app ``` ## Additional Options Appium provides [a lot of options](https://appium.io/docs/en/2.0/guides/caps/) to configure your test. Some important options that might help: **For Android:** - **appActivity** and **appPackage** : by default, Appium will try to extract the main Activity from your apk. If this fails, you can supply your own with these options. - **chromeOptions** : additional chromedriver options you can supply. - **otherApps** : a JSON array of other apps that need to be installed, in URL format. **For Android & iOS:** - **locale** : the language of the simulator/device (ex: `fr_CA`) This sets the locale on the entire device/simulator, except on physical iOS devices. For real iOS devices, we will pass `-AppleLocale` as argument to the app. - **newCommandTimeout** : How long (in seconds) Appium will wait for a new command from the client before assuming the client quit and ending the session ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/appium/nodejs/codeceptjs # Getting started with CodeceptJS [CodeceptJS](https://codecept.io/) is an end-to-end testing framework which supports Appium for automated mobile testing. It offers scenario driven testing, PageObjects support, beautiful reports and much more. Let's start with installing everything we need: ```bash npm i --save-dev codeceptjs ``` Now we need to set up CodeceptJS. Please use the command below to initialize a new project: ```bash npx codeceptjs init ``` This will ask a bunch of questions, and will create a sample test at the end of the process. ## Example test Now that you have everything set up, you can create a very simple test: ```javascript Feature('My First Mobile App Test'); Scenario('test sample calculator', ({ I }) => { I.seeAppIsInstalled("com.testingbot.sample"); I.click('~inputA'); I.fillField('~inputA', '10'); I.click('~inputB'); I.fillField('~inputB', '5'); I.see('15', '~sum'); }); ``` This simple test will test a calculator mobile app, it will fill in two numbers in two input fields and check the sum. Before we can run this test, we need to configure CodeceptJS to connect to TestingBot. ## Configuration You can configure CodeceptJS with a `codecept.conf.js` file: ```javascript exports.config = { tests: './*_test.js', output: './output', helpers: { Appium: { appiumV2: true, host: "hub.testingbot.com", port: 443, protocol: "https", path: "/wd/hub", user: "api_key", key: "api_secret", platform: "Android", app: "tb://...", desiredCapabilities: { platformName: "Android", deviceName: "Pixel 8", platformVersion: "14.0", automationName: "UiAutomator2", "tb:options": { name: "My first CodeceptJS test" } } }, }, include: { I: './steps_file.js' }, bootstrap: null, mocha: {}, name: 'codeceptjs-example', plugins: { retryFailedStep: { enabled: true }, screenshotOnFail: { enabled: true }, } } ``` To run the test, you can do: ```bash npx codeceptjs run --steps ``` This will start a Pixel 8 Emulator and run the test on our grid. You will see a test appear in your member dashboard on TestingBot, together with a video, log files and other assets generated by the test. ## Configuring capabilities To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before** ```javascript // codecept.conf.js exports.config = { tests: './*_test.js', output: './output', helpers: { Appium : {} } }; ``` **After** ```javascript // codecept.conf.js helpers: { Appium: { appiumV2: true, host: "hub.testingbot.com", port: 443, protocol: "https", path: "/wd/hub", user: "api_key", key: "api_secret", platform: "Android", app: "tb://...", desiredCapabilities: { platformName: "Android", deviceName: "Pixel 8", platformVersion: "14.0", automationName: "UiAutomator2", "tb:options": { name: "My first CodeceptJS test" } } }, }, ``` ### Specifying the device To let TestingBot know on which device you want to run your test on, you need to specify the deviceName and platformName. To see how to do this, please select a combination of deviceName and platformName in the drop-down menus below. ![OS selected](https://testingbot.com/assets/environments/svg/ios-383468addf160fa18d0e431f529420739d7f7d1206175f682fead627d2e99a52.svg) iOS 18.6 › iPhone 16 Loading environments... Please wait while we load the available browsers and platforms. ## Mark tests as passed/failed CodeceptJS allows you to mark tests as passed or failed, which is useful for debugging and reporting purposes. You can use the `I.see()` and `I.dontSee()` methods to check for specific elements or text on the page. If the element is not found, the test will fail. This is useful for checking if a specific element is present on the page, or if a specific text is visible. If the element or text is not found, the test will fail and you will see an error message in the report. ## Parallel Testing Please see [CodeceptJS parallel testing](https://codecept.io/parallel/#parallel-execution-by-workers) on how to run your tests in parallel. You can specify the concurrency by specifying a number of workers: ```bash npx codeceptjs run-workers 2 ``` or run multiple different devices with: ```bash npx codeceptjs run-multiple --all ``` ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Preparing your App You do not need to install any code or plugin to run a test. Below are some things that are necessary to successfully run a mobile test: **For Android:** - Please supply the URL to your `.apk` or `.aab` file. **Important:** the `.apk` file needs to be a x86 build (x86 ABI) for Android emulators. **For iOS Real Device:** - Please supply the URL to an `.ipa` file. **For iOS Simulator:** - Please supply the URL to a `.zip` file that contains your `.app` - The `.app` needs to be an iOS Simulator build: - Create a Simulator build: ```bash xcodebuild -sdk iphonesimulator -configuration Debug ``` - Zip the `.app` bundle: ```bash zip -r MyApp.zip MyApp.app ``` ## Additional Options Appium provides [a lot of options](https://appium.io/docs/en/2.0/guides/caps/) to configure your test. Some important options that might help: **For Android:** - **appActivity** and **appPackage** : by default, Appium will try to extract the main Activity from your apk. If this fails, you can supply your own with these options. - **chromeOptions** : additional chromedriver options you can supply. - **otherApps** : a JSON array of other apps that need to be installed, in URL format. **For Android & iOS:** - **locale** : the language of the simulator/device (ex: `fr_CA`) This sets the locale on the entire device/simulator, except on physical iOS devices. For real iOS devices, we will pass `-AppleLocale` as argument to the app. - **newCommandTimeout** : How long (in seconds) Appium will wait for a new command from the client before assuming the client quit and ending the session ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/appium/nodejs/webdriverio # WebDriverIO Automated App Testing First, make sure you have installed NodeJS. If you haven't, you can easily install it by [downloading NodeJS](https://nodejs.org/en/download/). See our [WebdriverIO examples repository](https://github.com/testingbot/webdriverio-examples) for some examples on how to use WebdriverIO with TestingBot. ## Installation TestingBot has its own [WebDriverIO service](https://webdriver.io/docs/testingbot-service.html) plugin which makes configuring your tests easier and which sends test meta-data back to TestingBot. Let's start with making sure `webdriverio` is installed: ```bash npm i --save-dev @wdio/cli @wdio/local-runner ``` You are now ready to run a WebDriver test with NodeJS on our grid. ## WebDriverIO TestingBot Service WebDriverIO contains a [TestingBot service](https://webdriver.io/docs/testingbot-service.html) which makes it easy to run tests with WebDriverIO on TestingBot. With this service, WebDriverIO will start a [TestingBot Tunnel](https://testingbot.com/support/tunnel) if required, and send back test meta-data (passed state, name, build-id, ...) In order to use the service you need to download it from NPM: ```bash npm install @wdio/testingbot-service --save-dev ``` ## Mocha Example To run Mocha tests on TestingBot, please follow these steps: ```bash npm install @wdio/testingbot-service --save-dev npm install @wdio/dot-reporter --save-dev npm install @wdio/mocha-framework --save-dev ``` Now we'll make a simple Mocha test that runs against a calculator app on a real Android device on TestingBot and verifies the sum of two numbers. Please create the following files: **wdio.conf.js** ```javascript exports.config = { /** * specify test files */ specs: [ './runner-specs/mocha.test.js' ], /** * capabilities */ capabilities: [{ platformName: 'Android', 'appium:automationName': 'UiAutomator2', 'appium:deviceName': 'Pixel 8', 'appium:platformVersion': '14.0', 'appium:app': 'https://testingbot.com/appium/sample.apk', 'tb:options': { name: 'My First App Test' } }], /** * test configurations */ logLevel: 'silent', waitforTimeout: 10000, connectionRetryTimeout: 480000, framework: 'mocha', services: [ ['testingbot'] ], user: 'api_key', key: 'api_secret', reporters: ['dot'], mochaOpts: { ui: 'bdd' } }; ``` **runner-specs/mocha.test.js** ```javascript const assert = require('assert') describe('calculator', function() { it('should calculate a sum', async () => { const inputA = await $('~inputA') await inputA.waitForDisplayed(5000) await inputA.click() try { await inputA.addValue('10') } catch (e) {} const inputB = await $('~inputB') await inputB.waitForDisplayed(5000) await inputB.click() try { await inputB.addValue('5') } catch (e) {} const sumElement = await $('~sum') const sum = await sumElement.getText() assert.equal(sum, '15') // 10 + 5 }) }) ``` Now we can run this mobile app test on TestingBot. The test will do a simple sum calculation and verify if the calculation was correct. To start the test, please run this command: ```bash npx wdio run wdio.conf.js ``` The test will now run on TestingBot. At the end of the test, the testname and success state will be available on TestingBot, together with a video and other meta-data. ## Uploading Your App Please see our [Upload Mobile App](https://testingbot.com/support/app-automate/help/upload) documentation to find out how to upload your app to TestingBot for testing. ## Specify Devices To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before:** ```javascript exports.config = { // ... services: ['selenium-standalone'], // ... }; ``` **After:** ```javascript exports.config = { // ... services: [ ['testingbot'] ], user: process.env.TB_KEY, key: process.env.TB_SECRET, // ... }; ``` To let TestingBot know on which device you want to run your test on, you need to specify the `appium:deviceName`, `appium:platformVersion`, `platformName` and other optional options in the capabilities field. To see how to do this, please select a combination of device type and device name in the drop-down menus below. ![OS selected](https://testingbot.com/assets/environments/svg/ios-383468addf160fa18d0e431f529420739d7f7d1206175f682fead627d2e99a52.svg) iOS 18.6 › iPhone 16 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a WebdriverIO test with our Tunnel: 1. Adjust your `wdio.conf.js` configuration to use `tbTunnel: true`. The [wdio-testingbot-service](https://webdriver.io/docs/testingbot-service/) will automatically download and use the tunnel for your tests. **wdio.conf.js** ```javascript exports.config = { services: [ ['testingbot', { tbTunnel: true }] ], user: 'api_key', key: 'api_secret' }; ``` ## Run Tests in Parallel Parallel testing allows you to run multiple tests at the same time, significantly reducing your total test execution time. You can run the same test across different device configurations, or run different tests on the same configuration, all simultaneously. TestingBot provides a large grid of browsers and devices, enabling you to run tests in parallel efficiently. This is one of our core features designed to help you test faster and ship more reliably. With WebDriverIO, you can enable parallel test execution by setting the `maxInstances` property in your `wdio.conf.js` file and specifying multiple `capabilities`. This defines how many instances of the same browser can run in parallel. We recommend setting this value to match the number of parallel sessions allowed by your TestingBot plan. ```javascript exports.config = { maxInstances: 5, // Adjust this based on your TestingBot plan capabilities: [{ platformName: 'Android', 'appium:automationName': 'UiAutomator2', 'appium:deviceName': 'Pixel 8', 'appium:platformVersion': '14.0', 'appium:app': 'https://testingbot.com/appium/sample.apk', 'tb:options': { name: 'My First App Test' } }, { platformName: 'Android', 'appium:automationName': 'UiAutomator2', 'appium:deviceName': 'Pixel 9', 'appium:platformVersion': '15.0', 'appium:app': 'https://testingbot.com/appium/sample.apk', 'tb:options': { name: 'My Second App Test' } }] }; ``` ### Queuing Every TestingBot plan includes a fixed number of parallel test slots. If you exceed this limit, additional tests will be automatically queued and executed as soon as slots become available, no tests will be dropped or skipped. ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. If you are using the [wdio-testingbot-service](https://webdriver.io/docs/testingbot-service/) then your tests will automatically report back meta-data to TestingBot (like test success, name, stacktrace, ...) ## Common errors Below are some common errors you might encounter when running WebDriverIO tests on TestingBot: - `UND_ERR_HEADERS_TIMEOUT` **Problem:** This error indicates that WebdriverIO did not receive a response in time. This might happen when requesting a device that is currently in use, or in case you are trying to run more tests in parallel than your plan allows and the tests have been [queued](https://testingbot.com#queuing) for a long time. **Solution:** You can try adding the `connectionRetryTimeout` in your WebDriverIO configuration with a high enough value: ```javascript connectionRetryTimeout: 480000 ``` ## Preparing your App You do not need to install any code or plugin to run a test. Below are some things that are necessary to successfully run a mobile test: **For Android:** - Please supply the URL to your `.apk` or `.aab` file. **Important:** the `.apk` file needs to be a x86 build (x86 ABI) for Android emulators. **For iOS Real Device:** - Please supply the URL to an `.ipa` file. **For iOS Simulator:** - Please supply the URL to a `.zip` file that contains your `.app` - The `.app` needs to be an iOS Simulator build: - Create a Simulator build: ```bash xcodebuild -sdk iphonesimulator -configuration Debug ``` - Zip the `.app` bundle: ```bash zip -r MyApp.zip MyApp.app ``` ## Additional Options Appium provides [a lot of options](https://appium.io/docs/en/2.0/guides/caps/) to configure your test. Some important options that might help: **For Android:** - **appActivity** and **appPackage** : by default, Appium will try to extract the main Activity from your apk. If this fails, you can supply your own with these options. - **chromeOptions** : additional chromedriver options you can supply. - **otherApps** : a JSON array of other apps that need to be installed, in URL format. **For Android & iOS:** - **locale** : the language of the simulator/device (ex: `fr_CA`) This sets the locale on the entire device/simulator, except on physical iOS devices. For real iOS devices, we will pass `-AppleLocale` as argument to the app. - **newCommandTimeout** : How long (in seconds) Appium will wait for a new command from the client before assuming the client quit and ending the session ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/appium/php # PHPUnit Automated App Testing See our [PHPUnit Mobile example repository](https://github.com/testingbot/php-phpunit-mobile-example) for a simple example on how to run PHPUnit tests for Mobile Apps on TestingBot. Let's start with making sure PHP is available on your system. **For Windows:** - You can download PHP for Windows from [PHP For Windows](https://windows.php.net/download/). - Run the installer and follow the setup wizard to install PHP. **For Linux:** ```bash sudo apt-get install curl libcurl3 libcurl3-dev php ``` **For macOS:** PHP should already be present on macOS by default. ## Install PHPUnit and php-webdriver The legacy `appium/php-client` and `phpunit/phpunit-selenium` packages are no longer maintained. The examples on this page use the actively-maintained [php-webdriver/webdriver](https://github.com/php-webdriver/php-webdriver) client, which speaks the W3C / Appium 2 protocol. - Make sure you have [Composer installed](https://getcomposer.org/). - Install PHPUnit and php-webdriver in your project: ## PHPUnit Example Real Devices Please see the example below. We're going to test our sample app on a Real Device. In this test we'll add values to two input fields and see if the calculator works. [Android](https://testingbot.com#) [iOS](https://testingbot.com#) ```php setCapability('platformName', 'Android'); $caps->setCapability('appium:deviceName', 'Galaxy S24'); $caps->setCapability('appium:platformVersion', '14.0'); $caps->setCapability('appium:automationName', 'UiAutomator2'); $caps->setCapability('appium:app', 'https://testingbot.com/appium/sample.apk'); $caps->setCapability('tb:options', [ 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'PHP Appium Example', 'realDevice' => true ]); $driver = RemoteWebDriver::create('https://hub.testingbot.com/wd/hub', $caps, 120000); $driver->findElement(AppiumBy::accessibilityId('inputA'))->sendKeys('10'); $driver->findElement(AppiumBy::accessibilityId('inputB'))->sendKeys('5'); echo $driver->findElement(AppiumBy::accessibilityId('sum'))->getText(); $driver->quit(); ``` ```php setCapability('platformName', 'iOS'); $caps->setCapability('appium:deviceName', 'iPhone 15'); $caps->setCapability('appium:platformVersion', '18.0'); $caps->setCapability('appium:automationName', 'XCUITest'); $caps->setCapability('appium:app', 'https://testingbot.com/appium/sample.ipa'); $caps->setCapability('tb:options', [ 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'PHP Appium Example', 'realDevice' => true ]); $driver = RemoteWebDriver::create('https://hub.testingbot.com/wd/hub', $caps, 120000); $driver->findElement(AppiumBy::accessibilityId('inputA'))->sendKeys('10'); $driver->findElement(AppiumBy::accessibilityId('inputB'))->sendKeys('5'); $driver->quit(); ``` ## PHPUnit Example Simulator/Emulator Please see the example below. We're going to test our sample app on an iOS Simulator or Android Emulator. In this test we'll add values to two input fields and see if the calculator works. [Android](https://testingbot.com#) [iOS](https://testingbot.com#) ```php setCapability('platformName', 'Android'); $caps->setCapability('appium:deviceName', 'Pixel 8'); $caps->setCapability('appium:platformVersion', '14.0'); $caps->setCapability('appium:automationName', 'UiAutomator2'); $caps->setCapability('appium:app', 'https://testingbot.com/appium/sample.apk'); $caps->setCapability('tb:options', [ 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'PHP Appium Example' ]); $driver = RemoteWebDriver::create('https://hub.testingbot.com/wd/hub', $caps, 120000); $driver->findElement(AppiumBy::accessibilityId('inputA'))->sendKeys('10'); $driver->findElement(AppiumBy::accessibilityId('inputB'))->sendKeys('5'); $driver->quit(); ``` ```php setCapability('platformName', 'iOS'); $caps->setCapability('appium:deviceName', 'iPhone 15'); $caps->setCapability('appium:platformVersion', '17.4'); $caps->setCapability('appium:automationName', 'XCUITest'); $caps->setCapability('appium:app', 'https://testingbot.com/appium/sample.zip'); $caps->setCapability('tb:options', [ 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'PHP Appium Example' ]); $driver = RemoteWebDriver::create('https://hub.testingbot.com/wd/hub', $caps, 120000); $driver->findElement(AppiumBy::accessibilityId('inputA'))->sendKeys('10'); $driver->findElement(AppiumBy::accessibilityId('inputB'))->sendKeys('5'); $driver->quit(); ``` ## Uploading Your App Please see our [Upload Mobile App](https://testingbot.com/support/app-automate/help/upload) documentation to find out how to upload your app to TestingBot for testing. ## Specify Browsers & Devices To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: ```php $driver = RemoteWebDriver::create('https://API_KEY:API_SECRET@hub.testingbot.com/wd/hub', $caps, 120000); ``` To let TestingBot know on which device you want to run your test on, you need to specify the `appium:deviceName`, `appium:platformVersion`, `platformName` and other optional options in the capabilities field. ```php $caps = new DesiredCapabilities(); $caps->setCapability('platformName', 'iOS'); $caps->setCapability('appium:deviceName', 'iPhone 15'); $caps->setCapability('appium:platformVersion', '17.4'); $caps->setCapability('appium:automationName', 'XCUITest'); $caps->setCapability('appium:app', 'tb://...'); ``` To see how to do this, please select a combination of device type and device name in the drop-down menus below. ![OS selected](https://testingbot.com/assets/environments/svg/ios-383468addf160fa18d0e431f529420739d7f7d1206175f682fead627d2e99a52.svg) iOS 18.6 › iPhone 16 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a PHP WebDriver test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```php setCapability('platformName', 'iOS'); $caps->setCapability('appium:deviceName', 'iPhone 15'); $caps->setCapability('appium:platformVersion', '17.4'); $caps->setCapability('appium:automationName', 'XCUITest'); $caps->setCapability('appium:app', 'https://testingbot.com/appium/sample.zip'); $caps->setCapability('tb:options', [ 'name' => 'PHP Appium Tunnel Example' ]); // Point to local tunnel instead of hub.testingbot.com $driver = RemoteWebDriver::create('http://localhost:4445/wd/hub', $caps, 120000); $driver->findElement(AppiumBy::accessibilityId('inputA'))->sendKeys('10'); $driver->findElement(AppiumBy::accessibilityId('inputB'))->sendKeys('5'); $driver->quit(); ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. To run tests in parallel, we recommend using [Paratest](https://github.com/brianium/paratest), which makes it very easy to run multiple PHPUnit tests simultaneously. ```bash vendor/bin/paratest -p 8 -f --phpunit=vendor/bin/phpunit WebdriverTest.php ``` The command above will run your tests in 8 separate processes (8 tests at once). ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. You can use our [PHP API client](https://github.com/testingbot/testingbot-php) to report back test results. ```php $api = new TestingBot\TestingBotAPI("API_KEY", "API_SECRET"); $api->updateJob($driver->getSessionID(), ['name' => 'mytest', 'success' => true]); ``` ## Preparing your App You do not need to install any code or plugin to run a test. Below are some things that are necessary to successfully run a mobile test: **For Android:** - Please supply the URL to your `.apk` or `.aab` file. **Important:** the `.apk` file needs to be a x86 build (x86 ABI) for Android emulators. **For iOS Real Device:** - Please supply the URL to an `.ipa` file. **For iOS Simulator:** - Please supply the URL to a `.zip` file that contains your `.app` - The `.app` needs to be an iOS Simulator build: - Create a Simulator build: ```bash xcodebuild -sdk iphonesimulator -configuration Debug ``` - Zip the `.app` bundle: ```bash zip -r MyApp.zip MyApp.app ``` ## Additional Options Appium provides [a lot of options](https://appium.io/docs/en/2.0/guides/caps/) to configure your test. Some important options that might help: **For Android:** - **appActivity** and **appPackage** : by default, Appium will try to extract the main Activity from your apk. If this fails, you can supply your own with these options. - **chromeOptions** : additional chromedriver options you can supply. - **otherApps** : a JSON array of other apps that need to be installed, in URL format. **For Android & iOS:** - **locale** : the language of the simulator/device (ex: `fr_CA`) This sets the locale on the entire device/simulator, except on physical iOS devices. For real iOS devices, we will pass `-AppleLocale` as argument to the app. - **newCommandTimeout** : How long (in seconds) Appium will wait for a new command from the client before assuming the client quit and ending the session ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/appium/proxy # Configure a device with a proxy You can use Appium or Selenium WebDriver to configure a proxy on the remote Android or iOS device, at the start of your test. This makes sure that all traffic on the device will go through the external proxy, which can be useful to do [geolocation testing](https://testingbot.com/support/app-automate/appium/options#geo) or latency testing. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby desired_caps = { 'platformName' => 'Android', 'appium:automationName' => 'UiAutomator2', 'appium:deviceName' => 'Pixel 9', 'appium:platformVersion' => '16.0', 'appium:app' => 'https://testingbot.com/appium/sample.apk', 'tb:options' => { 'realDevice' => true } } # Set up the proxy settings proxy = { 'proxyType' => 'manual', # Proxy type (manual or system) 'httpProxy' => 'http://your-proxy-server:8080', # Replace with your HTTP proxy URL 'sslProxy' => 'http://your-proxy-server:8080', # Replace with your SSL proxy URL 'noProxy' => 'localhost,127.0.0.1' # Optional: bypass proxy for specific hosts } # Add proxy to desired capabilities desired_caps['proxy'] = proxy # Initialize the Appium driver driver = Appium::Driver.new({ caps: desired_caps, appium_lib: { server_url: 'https://API_KEY:API_SECRET@hub.testingbot.com/wd/hub' } }, true) ``` ```java UiAutomator2Options options = new UiAutomator2Options(); // Set basic capabilities options.setPlatformName("Android"); options.setAutomationName("UiAutomator2"); options.setCapability("appium:deviceName", "Pixel 9"); options.setCapability("appium:platformVersion", "16.0"); options.setCapability("appium:app", "https://testingbot.com/appium/sample.apk"); // Set TestingBot-specific options Map tbOptions = new HashMap<>(); tbOptions.put("realDevice", true); options.setCapability("tb:options", tbOptions); // Set up proxy settings Proxy proxy = new Proxy(); proxy.setHttpProxy("http://your-proxy-server:8080"); // Replace with your HTTP proxy URL proxy.setSslProxy("http://your-proxy-server:8080"); // Replace with your SSL proxy URL // Apply the proxy settings to the capabilities options.setProxy(proxy); // Initialize the driver with TestingBot Appium server URL URL appiumServerUrl = new URL("https://hub.testingbot.com/wd/hub"); AndroidDriver driver = new AndroidDriver(appiumServerUrl, options); ``` ```php $capabilities = array( 'platformName' => 'Android', 'appium:automationName' => 'UiAutomator2', 'appium:app' => 'https://testingbot.com/appium/sample.apk', 'appium:deviceName' => 'Pixel 9', 'appium:platformVersion' => '16.0', 'tb:options' => array( 'realDevice' => true ), 'proxy' => array( 'proxyType' => 'manual', // Proxy type, "manual" for custom proxy settings 'httpProxy' => 'http://your-proxy-server:8080', // Replace with your HTTP proxy 'sslProxy' => 'http://your-proxy-server:8080', // Replace with your SSL proxy (HTTPS traffic) 'noProxy' => 'localhost,127.0.0.1' // Optional: addresses that should bypass the proxy ) ); // The proxy is declared in the W3C 'proxy' capability above, which configures the proxy on the device. // Do NOT pass a proxy as the 5th/6th args of RemoteWebDriver::create(); those set the client-to-server HTTP proxy and would break the connection to the hub. $web_driver = RemoteWebDriver::create('https://API_KEY:API_SECRET@hub.testingbot.com/wd/hub', $capabilities); ``` ```python capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "Pixel 9", "appium:platformVersion": "16.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "realDevice": True }, "proxy": { "proxyType": "manual", # Or "system" if you want to use system proxy "httpProxy": "http://your-proxy-server:8080", # Replace with your proxy server "sslProxy": "http://your-proxy-server:8080", # Replace with your proxy server for SSL "noProxy": "localhost,127.0.0.1" # Optional: specify addresses to bypass the proxy } } from appium import webdriver from appium.options.android import UiAutomator2Options options = UiAutomator2Options().load_capabilities(capabilities) driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options) ``` ```javascript const { remote } = require('webdriverio'); const capabilities = { hostname: 'hub.testingbot.com', port: 443, protocol: 'https', path: '/wd/hub', user: 'api_key', key: 'api_secret', capabilities: { 'platformName': 'Android', 'appium:automationName': 'UiAutomator2', 'appium:deviceName': 'Pixel 9', 'appium:platformVersion': '16.0', 'appium:app': 'https://testingbot.com/appium/sample.apk', 'tb:options': { 'name': 'My First App Test', 'realDevice': true }, 'proxy': { proxyType: 'manual', httpProxy: 'http://proxy-server:8080', // Replace with your proxy server URL sslProxy: 'http://proxy-server:8080', // If needed for SSL connections noProxy: '' // You can define URLs to bypass the proxy } } }; const driver = await remote(capabilities); ``` ```csharp AppiumOptions options = new AppiumOptions(); options.PlatformName = "Android"; options.AddAdditionalAppiumOption("appium:automationName", "UiAutomator2"); options.AddAdditionalAppiumOption("appium:deviceName", "Pixel 9"); options.AddAdditionalAppiumOption("appium:platformVersion", "16.0"); options.AddAdditionalAppiumOption("appium:app", "https://testingbot.com/appium/sample.apk"); // Set TestingBot-specific options var tbOptions = new Dictionary { ["realDevice"] = true }; options.AddAdditionalAppiumOption("tb:options", tbOptions); // Set up proxy using the Proxy class Proxy proxy = new Proxy(); proxy.HttpProxy = "http://your-proxy-server:8080"; // Replace with your proxy URL proxy.SslProxy = "http://your-proxy-server:8080"; // Replace with your proxy URL proxy.NoProxy = "localhost,127.0.0.1"; // Optional: bypass proxy for specific addresses // Apply the proxy to the Appium options options.Proxy = proxy; // Initialize the driver AndroidDriver driver = new AndroidDriver(new Uri("https://hub.testingbot.com/wd/hub"), options); ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/appium/python # Python Automated App Testing Before you can start testing apps with Python, let's start with making sure Python is available on your system. **For Windows:** - Download the latest python installer for Windows: [Python Releases for Windows](https://www.python.org/downloads/windows/) - Run the installer and follow the setup wizard to install Python. **For Linux/macOS:** - macOS, Ubuntu and most other Linux distros come with Python pre-installed. ## Installation With TestingBot you can easily run automated tests with Python against your mobile app. To get started, please install the following libraries: ```bash pip install Appium-Python-Client selenium ``` ### Real Device Example [Android](https://testingbot.com#) [iOS](https://testingbot.com#) Below is a Python Example, this will download and install our sample app (a calculator), add numbers to two input fields and show the sum of the two numbers: ```python from appium import webdriver from appium.options.common import AppiumOptions from appium.webdriver.common.appiumby import AppiumBy from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "Galaxy S24", "appium:platformVersion": "14.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", "realDevice": True } } options = AppiumOptions().load_capabilities(capabilities) driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options) inputA = WebDriverWait(driver, 30).until( EC.element_to_be_clickable((AppiumBy.ACCESSIBILITY_ID, "inputA")) ) inputA.send_keys("10") inputB = WebDriverWait(driver, 30).until( EC.element_to_be_clickable((AppiumBy.ACCESSIBILITY_ID, "inputB")) ) inputB.send_keys("5") sum_element = WebDriverWait(driver, 30).until( EC.element_to_be_clickable((AppiumBy.ACCESSIBILITY_ID, "sum")) ) assert sum_element.text == "15", "Sum should be 15" driver.quit() ``` ```python from appium import webdriver from appium.options.common import AppiumOptions from appium.webdriver.common.appiumby import AppiumBy from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC capabilities = { "platformName": "iOS", "appium:automationName": "XCUITest", "appium:deviceName": "iPhone 15", "appium:platformVersion": "18.0", "appium:app": "https://testingbot.com/appium/sample.ipa", "tb:options": { "key": "api_key", "secret": "api_secret", "realDevice": True } } options = AppiumOptions().load_capabilities(capabilities) driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options) inputA = WebDriverWait(driver, 30).until( EC.element_to_be_clickable((AppiumBy.ACCESSIBILITY_ID, "inputA")) ) inputA.send_keys("10") inputB = WebDriverWait(driver, 30).until( EC.element_to_be_clickable((AppiumBy.ACCESSIBILITY_ID, "inputB")) ) inputB.send_keys("5") driver.quit() ``` ### Simulator/Emulator Example [Android](https://testingbot.com#) [iOS](https://testingbot.com#) Below is a Python Example, this will download and install our sample app (a calculator), add numbers to two input fields and show the sum of the two numbers: ```python from appium import webdriver from appium.options.common import AppiumOptions from appium.webdriver.common.appiumby import AppiumBy from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "Pixel 8", "appium:platformVersion": "14.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret" } } options = AppiumOptions().load_capabilities(capabilities) driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options) inputA = WebDriverWait(driver, 30).until( EC.element_to_be_clickable((AppiumBy.ACCESSIBILITY_ID, "inputA")) ) inputA.send_keys("10") inputB = WebDriverWait(driver, 30).until( EC.element_to_be_clickable((AppiumBy.ACCESSIBILITY_ID, "inputB")) ) inputB.send_keys("5") sum_element = WebDriverWait(driver, 30).until( EC.element_to_be_clickable((AppiumBy.ACCESSIBILITY_ID, "sum")) ) assert sum_element.text == "15", "Sum should be 15" driver.quit() ``` ```python from appium import webdriver from appium.options.common import AppiumOptions from appium.webdriver.common.appiumby import AppiumBy from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC capabilities = { "platformName": "iOS", "appium:automationName": "XCUITest", "appium:deviceName": "iPhone 15", "appium:platformVersion": "17.4", "appium:app": "https://testingbot.com/appium/sample.zip", "tb:options": { "key": "api_key", "secret": "api_secret" } } options = AppiumOptions().load_capabilities(capabilities) driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options) inputA = WebDriverWait(driver, 30).until( EC.element_to_be_clickable((AppiumBy.ACCESSIBILITY_ID, "inputA")) ) inputA.send_keys("10") inputB = WebDriverWait(driver, 30).until( EC.element_to_be_clickable((AppiumBy.ACCESSIBILITY_ID, "inputB")) ) inputB.send_keys("5") driver.quit() ``` Make sure to always stop your test (`driver.quit()`), otherwise it will continue running, leading to a timeout. ## Uploading Your App Please see our [Upload Mobile App](https://testingbot.com/support/app-automate/help/upload) documentation to find out how to upload your app to TestingBot for testing. ## Specify Devices To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before:** ```python driver = webdriver.Remote("http://localhost:4723/wd/hub", options=AppiumOptions().load_capabilities(capabilities)) ``` **After:** ```python options = AppiumOptions().load_capabilities(capabilities) driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options) ``` To let TestingBot know on which device you want to run your test on, you need to specify the `appium:deviceName`, `appium:platformVersion`, `platformName` and other optional options in the capabilities field. ![OS selected](https://testingbot.com/assets/environments/svg/ios-383468addf160fa18d0e431f529420739d7f7d1206175f682fead627d2e99a52.svg) iOS 18.6 › iPhone 16 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a Python test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```python from appium import webdriver from appium.options.common import AppiumOptions from appium.webdriver.common.appiumby import AppiumBy from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC capabilities = { "platformName": "iOS", "appium:automationName": "XCUITest", "appium:deviceName": "iPhone 15", "appium:platformVersion": "17.4", "appium:app": "https://testingbot.com/appium/sample.zip", "tb:options": { "key": "api_key", "secret": "api_secret" } } # Point to local tunnel instead of hub.testingbot.com options = AppiumOptions().load_capabilities(capabilities) driver = webdriver.Remote("http://localhost:4445/wd/hub", options=options) inputA = WebDriverWait(driver, 30).until( EC.element_to_be_clickable((AppiumBy.ACCESSIBILITY_ID, "inputA")) ) inputA.send_keys("10") inputB = WebDriverWait(driver, 30).until( EC.element_to_be_clickable((AppiumBy.ACCESSIBILITY_ID, "inputB")) ) inputB.send_keys("5") driver.quit() ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different device configurations or run different tests all on the same device configuration. TestingBot has a large grid of devices, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. To run tests in parallel, we recommend using pytest with pytest-xdist, which makes it very easy to run multiple Python tests simultaneously: ```bash pip install pytest pytest-xdist ``` ```bash pytest -n ``` ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. You can use our [Python API client](https://github.com/testingbot/testingbotclient) to report back test results. ```python from testingbotclient import TestingBotClient tb = TestingBotClient("API_KEY", "API_SECRET") tb.tests.update_test(self.driver.session_id, name="My Test", passed=True, build="build-123") ``` ## Preparing your App You do not need to install any code or plugin to run a test. Below are some things that are necessary to successfully run a mobile test: **For Android:** - Please supply the URL to your `.apk` or `.aab` file. **Important:** the `.apk` file needs to be a x86 build (x86 ABI) for Android emulators. **For iOS Real Device:** - Please supply the URL to an `.ipa` file. **For iOS Simulator:** - Please supply the URL to a `.zip` file that contains your `.app` - The `.app` needs to be an iOS Simulator build: - Create a Simulator build: ```bash xcodebuild -sdk iphonesimulator -configuration Debug ``` - Zip the `.app` bundle: ```bash zip -r MyApp.zip MyApp.app ``` ## Additional Options Appium provides [a lot of options](https://appium.io/docs/en/2.0/guides/caps/) to configure your test. Some important options that might help: **For Android:** - **appActivity** and **appPackage** : by default, Appium will try to extract the main Activity from your apk. If this fails, you can supply your own with these options. - **chromeOptions** : additional chromedriver options you can supply. - **otherApps** : a JSON array of other apps that need to be installed, in URL format. **For Android & iOS:** - **locale** : the language of the simulator/device (ex: `fr_CA`) This sets the locale on the entire device/simulator, except on physical iOS devices. For real iOS devices, we will pass `-AppleLocale` as argument to the app. - **newCommandTimeout** : How long (in seconds) Appium will wait for a new command from the client before assuming the client quit and ending the session ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/appium/robotframework # Robot Framework Automated App Testing See our [Robot Framework app example repository](https://github.com/testingbot/robotframework-app-example) for a complete example on how to run Robot Framework mobile app tests on TestingBot. With [AppiumLibrary](https://github.com/serhatbolsu/robotframework-appiumlibrary) you can run [Appium](https://appium.io) tests with [Robot Framework](https://robotframework.org/) against native and hybrid mobile apps on Android and iOS. Robot Framework is a keyword-driven test automation framework that produces easy to read test cases and HTML reports. Combined with TestingBot's device cloud, you can run your Robot Framework app tests on a large range of real Android and iOS devices, Android emulators and iOS simulators. ## Installation Make sure you have Python and [pip](https://pip.pypa.io/en/stable/) installed, then install the required libraries: ```bash pip install robotframework robotframework-appiumlibrary Appium-Python-Client requests ``` To run a test, use the `robot` command. The examples below rely on two small Python helper libraries (`TestingBot.py` and `AppOptions.py`), see the [Helper Libraries](https://testingbot.com#helpers) section for their content, or clone the [example repository](https://github.com/testingbot/robotframework-app-example). ```bash PYTHONPATH=$PYTHONPATH:. robot test_android_emulator.robot ``` ## Android Examples Below are Robot Framework examples that download and install our sample Android app (a calculator), fill in two input fields and verify the sum. Switch between running on an Android emulator or on a real Android device. [Emulator](https://testingbot.com#) [Real Device](https://testingbot.com#) Save the example below in a `test_android_emulator.robot` file: ```python ***Settings*** Library AppiumLibrary Library TestingBot Library AppOptions Test Setup Open test app Test Teardown Close test app ***Test Cases*** Android Sample App Sum Test Wait Until Page Contains Element accessibility_id=inputA timeout=30s Input Text accessibility_id=inputA 10 Wait Until Page Contains Element accessibility_id=inputB timeout=30s Input Text accessibility_id=inputB 5 Wait Until Page Contains Element accessibility_id=sum timeout=30s Element Text Should Be accessibility_id=sum 15 ***Keywords*** Open test app ${options}= Get App Options Android # W3C + Appium capabilities for a TestingBot Android emulator Call Method ${options} set_capability platformName Android Call Method ${options} set_capability appium:deviceName Pixel.* Call Method ${options} set_capability appium:platformVersion 16.0 Call Method ${options} set_capability appium:automationName UiAutomator2 Call Method ${options} set_capability appium:app https://testingbot.com/appium/sample.apk # TestingBot capabilities ${tb_options}= Create Dictionary key=api_key secret=api_secret name=Android Emulator App Test build=MyAppBuild Call Method ${options} set_capability tb:options ${tb_options} Open Testingbot App ${options} Close test app Report Testingbot Status ${SUITE_NAME} | ${TEST_NAME} ${TEST_STATUS} Close Application ``` ```bash PYTHONPATH=$PYTHONPATH:. robot test_android_emulator.robot ``` Save the example below in a `test_android_physical.robot` file: ```python ***Settings*** Library AppiumLibrary Library TestingBot Library AppOptions Test Setup Open test app Test Teardown Close test app ***Test Cases*** Android Sample App Sum Test Wait Until Page Contains Element accessibility_id=inputA timeout=30s Input Text accessibility_id=inputA 10 Wait Until Page Contains Element accessibility_id=inputB timeout=30s Input Text accessibility_id=inputB 5 Wait Until Page Contains Element accessibility_id=sum timeout=30s Element Text Should Be accessibility_id=sum 15 ***Keywords*** Open test app ${options}= Get App Options Android # W3C + Appium capabilities for a TestingBot Android real device Call Method ${options} set_capability platformName Android Call Method ${options} set_capability appium:deviceName Galaxy S24 Call Method ${options} set_capability appium:platformVersion 14.0 Call Method ${options} set_capability appium:automationName UiAutomator2 Call Method ${options} set_capability appium:app https://testingbot.com/appium/sample.apk # TestingBot capabilities, realDevice=True routes the test to a physical device ${tb_options}= Create Dictionary key=api_key secret=api_secret name=Android Physical App Test build=MyAppBuild realDevice=True Call Method ${options} set_capability tb:options ${tb_options} Open Testingbot App ${options} Close test app Report Testingbot Status ${SUITE_NAME} | ${TEST_NAME} ${TEST_STATUS} Close Application ``` ```bash PYTHONPATH=$PYTHONPATH:. robot test_android_physical.robot ``` ## iOS Examples Below are Robot Framework examples for our iOS sample app. Switch between running on an iOS simulator or on a real iPhone/iPad. [Simulator](https://testingbot.com#) [Real Device](https://testingbot.com#) Save the example below in a `test_ios_simulator.robot` file: ```python ***Settings*** Library AppiumLibrary Library TestingBot Library AppOptions Test Setup Open test app Test Teardown Close test app ***Test Cases*** iOS Sample App Sum Test Wait Until Page Contains Element accessibility_id=inputA timeout=30s Input Text accessibility_id=inputA 10 Wait Until Page Contains Element accessibility_id=inputB timeout=30s Input Text accessibility_id=inputB 5 ***Keywords*** Open test app ${options}= Get App Options iOS # W3C + Appium capabilities for a TestingBot iOS simulator Call Method ${options} set_capability platformName iOS Call Method ${options} set_capability appium:deviceName iPhone.* Call Method ${options} set_capability appium:platformVersion 26.0 Call Method ${options} set_capability appium:automationName XCUITest Call Method ${options} set_capability appium:app https://testingbot.com/appium/sample.zip # TestingBot capabilities ${tb_options}= Create Dictionary key=api_key secret=api_secret name=iOS Simulator App Test build=MyAppBuild Call Method ${options} set_capability tb:options ${tb_options} Open Testingbot App ${options} Close test app Report Testingbot Status ${SUITE_NAME} | ${TEST_NAME} ${TEST_STATUS} Close Application ``` ```bash PYTHONPATH=$PYTHONPATH:. robot test_ios_simulator.robot ``` Save the example below in a `test_ios_physical.robot` file: ```python ***Settings*** Library AppiumLibrary Library TestingBot Library AppOptions Test Setup Open test app Test Teardown Close test app ***Test Cases*** iOS Sample App Sum Test Wait Until Page Contains Element accessibility_id=inputA timeout=30s Input Text accessibility_id=inputA 10 Wait Until Page Contains Element accessibility_id=inputB timeout=30s Input Text accessibility_id=inputB 5 ***Keywords*** Open test app ${options}= Get App Options iOS # W3C + Appium capabilities for a TestingBot iOS real device Call Method ${options} set_capability platformName iOS Call Method ${options} set_capability appium:deviceName iPhone.* Call Method ${options} set_capability appium:platformVersion 26.* Call Method ${options} set_capability appium:automationName XCUITest Call Method ${options} set_capability appium:app https://testingbot.com/appium/sample.ipa # TestingBot capabilities, realDevice=True routes the test to a physical device ${tb_options}= Create Dictionary key=api_key secret=api_secret name=iOS Physical App Test build=MyAppBuild realDevice=True Call Method ${options} set_capability tb:options ${tb_options} Open Testingbot App ${options} Close test app Report Testingbot Status ${SUITE_NAME} | ${TEST_NAME} ${TEST_STATUS} Close Application ``` ```bash PYTHONPATH=$PYTHONPATH:. robot test_ios_physical.robot ``` Make sure to always close the session in your test teardown (`Close Application`), otherwise the session will keep running until TestingBot's idle timeout. ## Helper Libraries The examples above import two small Python helper libraries, `AppOptions` and `TestingBot`, that encapsulate driver creation and test status reporting. You can copy the snippets below, or grab them from the [example repository](https://github.com/testingbot/robotframework-app-example). Save the following in a `AppOptions.py` file: ```python from appium.options.android import UiAutomator2Options from appium.options.ios import XCUITestOptions def get_app_options(platform): if platform.lower() == 'android': return UiAutomator2Options() elif platform.lower() == 'ios': return XCUITestOptions() else: raise ValueError(f"Unsupported platform: {platform}") ``` Save the following in a `TestingBot.py` file: ```python import os import requests from appium import webdriver from robot.libraries.BuiltIn import BuiltIn TB_KEY = os.environ.get('TB_KEY', 'api_key') TB_SECRET = os.environ.get('TB_SECRET', 'api_secret') HUB_URL = f'https://{TB_KEY}:{TB_SECRET}@hub.testingbot.com/wd/hub' def open_testingbot_app(options): driver = webdriver.Remote(HUB_URL, options=options) appium = BuiltIn().get_library_instance('AppiumLibrary') appium._cache.register(driver) def report_testingbot_status(name, status): appium = BuiltIn().get_library_instance('AppiumLibrary') session_id = appium._current_application().session_id payload = {'test[name]': name, 'test[success]': int(status == 'PASS')} url = f'https://api.testingbot.com/v1/tests/{session_id}' response = requests.put(url, data=payload, auth=(TB_KEY, TB_SECRET)) assert response.status_code == 200, response.text ``` ## Uploading Your App Please see our [Upload Mobile App](https://testingbot.com/support/app-automate/help/upload) documentation to find out how to upload your `.apk`, `.ipa` or `.zip` to TestingBot and reference it from the `appium:app` capability. ## Specify Devices To let TestingBot know which device to run your test on, set the `platformName`, `appium:deviceName` and `appium:platformVersion` capabilities. Pick a combination below to see the matching snippet: ![OS selected](https://testingbot.com/assets/environments/svg/ios-383468addf160fa18d0e431f529420739d7f7d1206175f682fead627d2e99a52.svg) iOS 18.6 › iPhone 16 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal apps or backends. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information. 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start it: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. In your `TestingBot.py` helper, point `HUB_URL` at the tunnel instead of `hub.testingbot.com`. Assuming the tunnel runs on the same machine as your tests, use `http://localhost:4445/wd/hub` (4445 is the default tunnel port). ```python HUB_URL = 'http://localhost:4445/wd/hub' ``` Your test will then route securely through the tunnel to TestingBot and back. ## Run tests in Parallel You can run multiple Robot Framework tests in parallel using the [Pabot](https://pabot.org/) library. Install Pabot: ```bash pip install -U robotframework-pabot ``` Run your tests with `pabot` instead of `robot`: ```bash PYTHONPATH=$PYTHONPATH:. pabot --processes 2 test_android_emulator.robot test_ios_simulator.robot ``` Some recommendations when running tests in parallel: - Make sure each test case is independent and does not rely on shared state. - Set `--processes` to the number of parallel sessions your TestingBot plan allows. - Consider using `--processtimeout` to set a timeout for each process. ### Queuing Every plan comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and start them as soon as slots become available. ## Preparing your App You do not need to install any code or plugin to run a test. Below are some things that are necessary to successfully run a mobile test: **For Android:** - Please supply the URL to your `.apk` or `.aab` file. **Important:** the `.apk` file needs to be a x86 build (x86 ABI) for Android emulators. **For iOS Real Device:** - Please supply the URL to an `.ipa` file. **For iOS Simulator:** - Please supply the URL to a `.zip` file that contains your `.app` - The `.app` needs to be an iOS Simulator build: - Create a Simulator build: ```bash xcodebuild -sdk iphonesimulator -configuration Debug ``` - Zip the `.app` bundle: ```bash zip -r MyApp.zip MyApp.app ``` ## Additional Options Appium provides [a lot of options](https://appium.io/docs/en/2.0/guides/caps/) to configure your test. Some important options that might help: **For Android:** - **appActivity** and **appPackage** : by default, Appium will try to extract the main Activity from your apk. If this fails, you can supply your own with these options. - **chromeOptions** : additional chromedriver options you can supply. - **otherApps** : a JSON array of other apps that need to be installed, in URL format. **For Android & iOS:** - **locale** : the language of the simulator/device (ex: `fr_CA`) This sets the locale on the entire device/simulator, except on physical iOS devices. For real iOS devices, we will pass `-AppleLocale` as argument to the app. - **newCommandTimeout** : How long (in seconds) Appium will wait for a new command from the client before assuming the client quit and ending the session ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/appium/ruby # Ruby Automated App Testing Let's start with making sure Ruby is available on your system. **For Windows:** - You can download Ruby for Windows from [RubyInstaller](https://rubyinstaller.org/) - Run the installer and follow the setup wizard to install Ruby. **For Linux/macOS:** Download [RVM](https://rvm.io/) and install Ruby: ```bash curl -L https://get.rvm.io | bash -s stable --ruby ``` ## Installation With TestingBot you can easily run your automated app tests with Ruby. Please install the following gems: ```bash gem install appium_lib gem install selenium-webdriver ``` Please see the example below. We're going to run a test on our sample app (calculator) where we'll add values to two input fields. ### Real Device Testing [Android](https://testingbot.com#) [iOS](https://testingbot.com#) ```ruby require 'appium_lib' caps = { "platformName" => "Android", "appium:automationName" => "UiAutomator2", "appium:deviceName" => "Galaxy S24", "appium:platformVersion" => "14.0", "appium:app" => "https://testingbot.com/appium/sample.apk", "tb:options" => { "name" => "Ruby Appium Example", "realDevice" => true } } appium_driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "https://API_KEY:API_SECRET@hub.testingbot.com/wd/hub" }}, true) driver = appium_driver.start_driver wait = Selenium::WebDriver::Wait.new(:timeout => 30) wait.until { driver.find_element(:accessibility_id, "inputA").displayed? } driver.find_element(:accessibility_id, "inputA").send_keys 5 driver.find_element(:accessibility_id, "inputB").send_keys 10 result = driver.find_element(:accessibility_id, "sum") if (!result.nil?) && (result.text.eql? "15") puts "Test Passed" else puts "Test Failed" end driver.quit ``` ```ruby require 'appium_lib' caps = { "platformName" => "iOS", "appium:automationName" => "XCUITest", "appium:deviceName" => "iPhone 15", "appium:platformVersion" => "18.0", "appium:app" => "https://testingbot.com/appium/sample.ipa", "tb:options" => { "name" => "Ruby Appium Example", "realDevice" => true } } appium_driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "https://API_KEY:API_SECRET@hub.testingbot.com/wd/hub" }}, true) driver = appium_driver.start_driver wait = Selenium::WebDriver::Wait.new(:timeout => 30) driver.find_element(:accessibility_id, "inputA").send_keys 5 driver.find_element(:accessibility_id, "inputB").send_keys 10 driver.quit ``` ### Simulator/Emulator Testing [Android](https://testingbot.com#) [iOS](https://testingbot.com#) ```ruby require 'appium_lib' caps = { "platformName" => "Android", "appium:automationName" => "UiAutomator2", "appium:deviceName" => "Pixel 8", "appium:platformVersion" => "14.0", "appium:app" => "https://testingbot.com/appium/sample.apk", "tb:options" => { "name" => "Ruby Appium Example" } } appium_driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "https://API_KEY:API_SECRET@hub.testingbot.com/wd/hub" }}, true) driver = appium_driver.start_driver wait = Selenium::WebDriver::Wait.new(:timeout => 30) wait.until { driver.find_element(:accessibility_id, "inputA").displayed? } driver.find_element(:accessibility_id, "inputA").send_keys 5 driver.find_element(:accessibility_id, "inputB").send_keys 10 result = driver.find_element(:accessibility_id, "sum") if (!result.nil?) && (result.text.eql? "15") puts "Test Passed" else puts "Test Failed" end driver.quit ``` ```ruby require 'appium_lib' caps = { "platformName" => "iOS", "appium:automationName" => "XCUITest", "appium:deviceName" => "iPhone 15", "appium:platformVersion" => "17.4", "appium:app" => "https://testingbot.com/appium/sample.zip", "tb:options" => { "name" => "Ruby Appium Example" } } appium_driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "https://API_KEY:API_SECRET@hub.testingbot.com/wd/hub" }}, true) driver = appium_driver.start_driver wait = Selenium::WebDriver::Wait.new(:timeout => 30) driver.find_element(:accessibility_id, "inputA").send_keys 5 driver.find_element(:accessibility_id, "inputB").send_keys 10 driver.quit ``` Make sure to always stop your test (`driver.quit`), otherwise it will continue running, leading to a timeout. ## Uploading Your App Please see our [Upload Mobile App](https://testingbot.com/support/app-automate/help/upload) documentation to find out how to upload your app to TestingBot for testing. ## Specify Devices To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote devices. If the test was running on your local machine or network, you can simply change your existing test like this: **Before:** ```ruby appium_driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "http://localhost:4723/wd/hub" }}, true) ``` **After:** ```ruby appium_driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "https://API_KEY:API_SECRET@hub.testingbot.com/wd/hub" }}, true) ``` To let TestingBot know on which device you want to run your test on, you need to specify the `appium:deviceName`, `appium:platformVersion`, `platformName` and other optional options in the capabilities field. To see how to do this, please select a combination of device type and device name in the drop-down menus below. ![OS selected](https://testingbot.com/assets/environments/svg/ios-383468addf160fa18d0e431f529420739d7f7d1206175f682fead627d2e99a52.svg) iOS 18.6 › iPhone 16 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a Ruby WebDriver test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```ruby require 'appium_lib' caps = { "platformName" => "iOS", "appium:automationName" => "XCUITest", "appium:deviceName" => "iPhone 15", "appium:platformVersion" => "17.4", "appium:app" => "https://testingbot.com/appium/sample.zip", "tb:options" => { "name" => "single_test", "build" => "Ruby Appium Sample" } } appium_driver = Appium::Driver.new({ caps: caps, appium_lib: { # Point to local tunnel instead of hub.testingbot.com server_url: "http://localhost:4445/wd/hub" }}, true) driver = appium_driver.start_driver wait = Selenium::WebDriver::Wait.new(:timeout => 30) driver.find_element(:accessibility_id, "inputA").send_keys 5 driver.find_element(:accessibility_id, "inputB").send_keys 10 driver.quit ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different device configurations or run different tests all on the same device configuration. TestingBot has a large grid of devices, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. To get started, please see the examples on [https://github.com/grosser/parallel\_tests](https://github.com/grosser/parallel_tests) ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. You can use our [Ruby API client](https://github.com/testingbot/testingbot_ruby) to report back test results. ```ruby api = TestingBot::Api.new(key, secret) api.update_test(driver.session_id, { :name => new_name, :success => true }) ``` ## Preparing your App You do not need to install any code or plugin to run a test. Below are some things that are necessary to successfully run a mobile test: **For Android:** - Please supply the URL to your `.apk` or `.aab` file. **Important:** the `.apk` file needs to be a x86 build (x86 ABI) for Android emulators. **For iOS Real Device:** - Please supply the URL to an `.ipa` file. **For iOS Simulator:** - Please supply the URL to a `.zip` file that contains your `.app` - The `.app` needs to be an iOS Simulator build: - Create a Simulator build: ```bash xcodebuild -sdk iphonesimulator -configuration Debug ``` - Zip the `.app` bundle: ```bash zip -r MyApp.zip MyApp.app ``` ## Additional Options Appium provides [a lot of options](https://appium.io/docs/en/2.0/guides/caps/) to configure your test. Some important options that might help: **For Android:** - **appActivity** and **appPackage** : by default, Appium will try to extract the main Activity from your apk. If this fails, you can supply your own with these options. - **chromeOptions** : additional chromedriver options you can supply. - **otherApps** : a JSON array of other apps that need to be installed, in URL format. **For Android & iOS:** - **locale** : the language of the simulator/device (ex: `fr_CA`) This sets the locale on the entire device/simulator, except on physical iOS devices. For real iOS devices, we will pass `-AppleLocale` as argument to the app. - **newCommandTimeout** : How long (in seconds) Appium will wait for a new command from the client before assuming the client quit and ending the session ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/appium/test-app-upgrades # Test App Upgrades/Mid-Session App Installations As app developers, we frequently release new versions of our applications. In most cases, users upgrade from an older version rather than installing the app from scratch. This transition often requires migrating existing user data to ensure the app functions correctly, making upgrade testing a crucial step in development. By thoroughly testing upgrades, we can confirm that the app continues to work as expected post-update. App upgrades aren't the only scenario where installing apps during a running session is necessary. Some applications may depend on other apps to function properly. If your app interacts with another application, such as pulling data or integrating with its features, it's essential to verify that these dependencies remain intact after an upgrade. Additionally, there are cases where you might need to delete and reinstall an app during a session. For example, if you want to test how the app behaves when a user uninstalls and reinstalls it, the mid-session install feature can facilitate this process. ## (Re-)Installing apps during a test Before you can install a new app, or a new version of the app, we encourage you to first upload your `.apk/.ipa/.zip` file to [TestingBot Storage](https://testingbot.com/support/app-automate/help/upload). Alternatively, you can also specify a URL to an `.apk/.ipa/.zip` file that is publicly accessible. During your test, you can use the `mobile:installApp` command to install the app on the device. [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) [Ruby](https://testingbot.com#) [C#](https://testingbot.com#) ```java driver.executeScript("mobile: installApp", ImmutableMap.of("appPath", "tb://.apk|ipa|zip")); ``` ```javascript await driver.execute('mobile: installApp', {"appPath": "tb://.apk|ipa|zip"}); ``` ```python driver.execute_script('mobile: installApp', {"appPath": "tb://.apk|ipa|zip"}) ``` ```ruby driver.execute_script('mobile: installApp', {"appPath" => "tb://.apk|ipa|zip"}) ``` ```csharp driver.ExecuteScript("mobile: installApp", new Dictionary { { "appPath", "tb://.apk|ipa|zip" } }); ``` The `mobile: installApp` command returns no payload on success, Appium will throw an exception if the install fails. Use `mobile: queryAppState` (Android) or `mobile: isAppInstalled` to verify the install, and read the version from the app itself if needed. Appium will not automatically launch the app after you installed it during the test. You can launch the app manually using the following commands after installation: - **Android:** `mobile: startActivity` - **iOS:** `mobile: activateApp` (`mobile: launchApp` is deprecated) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) [Ruby](https://testingbot.com#) [C#](https://testingbot.com#) ```java // Android driver.executeScript("mobile: startActivity", ImmutableMap.of("intent", "com.testingbot.demo/.MainActivity")); // iOS driver.executeScript("mobile: activateApp", ImmutableMap.of("bundleId", "com.testingbot.demo")); ``` ```javascript // Android await driver.execute('mobile: startActivity', {intent: 'com.testingbot.demo/.MainActivity'}); // iOS await driver.execute('mobile: activateApp', {bundleId: 'com.testingbot.demo'}); ``` ```python # Android driver.execute_script("mobile: startActivity", {"intent": "com.testingbot.demo/.MainActivity"}) # iOS driver.execute_script("mobile: activateApp", {"bundleId": "com.testingbot.demo"}) ``` ```ruby # Android @driver.execute_script("mobile: startActivity", {"intent" => "com.testingbot.demo/.MainActivity"}) # iOS @driver.execute_script("mobile: activateApp", {"bundleId" => "com.testingbot.demo"}) ``` ```csharp // Android driver.ExecuteScript("mobile: startActivity", new Dictionary { { "intent", "com.testingbot.demo/.MainActivity" } }); // iOS driver.ExecuteScript("mobile: activateApp", new Dictionary { { "bundleId", "com.testingbot.demo" } }); ``` ## Removing your app during a test To remove your app during a test, you can use the [`mobile: removeApp`](https://github.com/appium/appium-uiautomator2-driver?tab=readme-ov-file#mobile-removeapp) command. [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) [Ruby](https://testingbot.com#) [C#](https://testingbot.com#) ```java // Android driver.executeScript("mobile: removeApp", ImmutableMap.of("appId", "com.testingbot.demo")); // iOS driver.executeScript("mobile: removeApp", ImmutableMap.of("bundleId", "com.testingbot.demo")); ``` ```javascript // Android await driver.execute('mobile: removeApp', {appId: 'com.testingbot.demo'}); // iOS await driver.execute('mobile: removeApp', {bundleId: 'com.testingbot.demo'}); ``` ```python # Android driver.execute_script("mobile: removeApp", {"appId": "com.testingbot.demo"}) # iOS driver.execute_script("mobile: removeApp", {"bundleId": "com.testingbot.demo"}) ``` ```ruby # Android @driver.execute_script("mobile: removeApp", {"appId" => "com.testingbot.demo"}) # iOS @driver.execute_script("mobile: removeApp", {"bundleId" => "com.testingbot.demo"}) ``` ```csharp // Android driver.ExecuteScript("mobile: removeApp", new Dictionary { { "appId", "com.testingbot.demo" } }); // iOS driver.ExecuteScript("mobile: removeApp", new Dictionary { { "bundleId", "com.testingbot.demo" } }); ``` ## Other app commands Appium supports more commands to handle apps during a test, such as: ### mobile: terminateApp This will forcefully stop the app (specified by bundleId or appId) on the device. ### mobile: isAppInstalled Verifies if the app (specified by bundleId or appId) is on the device. Returns a boolean true or false. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/appium/upload-files # Upload files to physical devices TestingBot provides a feature that allows you to specify one or more files that need to be uploaded to a physical iOS or Android device, prior to running your Appium test. This functionality allows you to run tests that require media files (photos and videos) from the iOS or Android photo gallery, or documents (PDFs, spreadsheets, any other file type) from the iOS Files app. There are three options to use this feature, depending on your usecase. We suggest using TestingBot's custom capabilities, which place the files on the device before your session starts. - [TestingBot Upload Capability](https://testingbot.com#testingbot) (`uploadMedia`: photos and videos into the gallery) - [Upload documents to the iOS Files app](https://testingbot.com#files) (`uploadFiles`: any file type) - [Appium Push and Pull](https://testingbot.com#appium) ## TestingBot Upload Capability To use this functionality, you can use the `uploadMedia` capability, specifying an array of URLs to image files or video files. You can specify URLs to publicly downloadable media files, or you can use TestingBot Storage to upload the files to TestingBot and use the `tb://` URLs. ### Upload files to TestingBot [cURL](https://testingbot.com#) ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/storage" \ -F "file=@/path/to/app/file/media.png" ``` The API response will include a `tb://` URL, which you can use in the `uploadMedia` capability. ```json { "app_url": "tb://edb4f4f3ed95a4f46714f3df" } ``` Uploads to our TestingBot Storage are automatically deleted after 60 days. ### Add the capability to your Appium script [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) [Ruby](https://testingbot.com#) [C#](https://testingbot.com#) ```java Map tbOptions = new HashMap<>(); tbOptions.put("uploadMedia", new String[]{"tb://edb4f4f3ed95a4f46714f3df"}); options.setCapability("tb:options", tbOptions); ``` ```javascript const capabilities = { 'tb:options': { 'uploadMedia': ['tb://edb4f4f3ed95a4f46714f3df'] } } ``` ```python capabilities = { "tb:options" : { "uploadMedia" : ["tb://edb4f4f3ed95a4f46714f3df"] } } ``` ```ruby capabilities = { "tb:options" => { "uploadMedia" => ["tb://edb4f4f3ed95a4f46714f3df"] } } ``` ```csharp AppiumOptions options = new AppiumOptions(); var tbOptions = new Dictionary(); tbOptions.Add("uploadMedia", new string[] { "tb://edb4f4f3ed95a4f46714f3df" }); options.AddAdditionalAppiumOption("tb:options", tbOptions); ``` TestingBot will add these files to the Android or iOS device. The metadata of the gallery on the device will be updated automatically, which means the media file will immediately appear in the device's gallery. - **Android:** the media files will be added to the Default Gallery app. `/sdcard/Pictures` for images and `/sdcard/Movies` for videos. - **iOS:** the media files will be added to the iOS Camera Roll: `com.apple.mobileslideshow` ## Upload documents to the iOS Files app For files that are not photos or videos, use the `uploadFiles` capability with an array of URLs. Any file type works: PDFs, spreadsheets, text files, archives, and so on. As with `uploadMedia`, the URLs can point to publicly downloadable files or to [TestingBot Storage](https://testingbot.com#upload) `tb://` URLs. Before your session starts, TestingBot downloads the files onto the physical iOS device. They appear in the **Files** app under **On My iPhone** (or **On My iPad** ) in a folder named **TestingBot** , each file keeping the name it has in the URL. Any app that uses the standard iOS document picker can open them from there, for example to test a file upload in your app. [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) [Ruby](https://testingbot.com#) [C#](https://testingbot.com#) ```java Map tbOptions = new HashMap<>(); tbOptions.put("uploadFiles", new String[]{"tb://edb4f4f3ed95a4f46714f3df", "https://example.com/contract.pdf"}); options.setCapability("tb:options", tbOptions); ``` ```javascript const capabilities = { 'tb:options': { 'uploadFiles': ['tb://edb4f4f3ed95a4f46714f3df', 'https://example.com/contract.pdf'] } } ``` ```python capabilities = { "tb:options" : { "uploadFiles" : ["tb://edb4f4f3ed95a4f46714f3df", "https://example.com/contract.pdf"] } } ``` ```ruby capabilities = { "tb:options" => { "uploadFiles" => ["tb://edb4f4f3ed95a4f46714f3df", "https://example.com/contract.pdf"] } } ``` ```csharp AppiumOptions options = new AppiumOptions(); var tbOptions = new Dictionary(); tbOptions.Add("uploadFiles", new string[] { "tb://edb4f4f3ed95a4f46714f3df", "https://example.com/contract.pdf" }); options.AddAdditionalAppiumOption("tb:options", tbOptions); ``` - **Location:** Files app → Browse → On My iPhone → **TestingBot**. - **Availability:** physical iOS and iPadOS devices. For Android, use `uploadMedia` or Appium's [pushFile](https://testingbot.com#appium). - **Combining:** `uploadMedia` and `uploadFiles` can be used together in the same session. - **Cleanup:** the folder is removed when your session ends; nothing is left on the device. ## Appium Push and Pull Appium provides both [pushFile](https://appium.io/docs/en/2.0/reference/interfaces/appium_types.ExternalDriver/#pushfile) and [pullFile](https://appium.io/docs/en/2.0/reference/interfaces/appium_types.ExternalDriver/#pullfile) functionality. This is supported since [Appium Version](https://testingbot.com/support/app-automate/appium/options#appiumVersion) 1.15.0. With this approach, the file is pushed when the test has already started. Pushing media files to the iOS or Android device does not guarantee the updating of the metadata in the iOS or Android gallery, which means your media file might not appear (immediately). ### Android Example Push a file to the SD Card of the Android device: [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) [Ruby](https://testingbot.com#) [C#](https://testingbot.com#) ```java // Push a file to the sdcard driver.pushFile("/sdcard/Download/sample.png", new File(System.getProperty("user.home") + "/sample.png")); // Pull file from sdcard byte[] fileBase64 = driver.pullFile("/sdcard/Download/sample.png"); ``` ```javascript // Push file - WebdriverIO syntax const data = Buffer.from("TestingBot").toString('base64'); await driver.pushFile('/sdcard/Download/sample.txt', data); // Pull file WebdriverIO example const fileData = await driver.pullFile('/sdcard/Download/sample.txt'); ``` ```python # Push a file dest_path = '/sdcard/Download/sample.txt' driver.push_file(dest_path, 'TestingBot'.encode("utf-8")) # Pull file file_base64 = driver.pull_file(dest_path) ``` ```ruby # Push a file driver.push_file('/sdcard/Download/sample.png', File.read('/Users/testingbot/Desktop/sample.png')) # Pull file driver.pull_file('/sdcard/Download/sample.png') ``` ```csharp // Push a file driver.PushFile("/sdcard/Download/sample.png", new FileInfo("/Users/testingbot/Desktop/sample.png")); // Pull file byte[] fileBase64 = driver.PullFile("/sdcard/Download/sample.png"); ``` ### iOS Example Before using the pull and push functionality, you'll need to ensure that: - Your iOS app must have set the `UIFileSharingEnabled` key to `true` in the `Info.plist` file, which enables file sharing. - Alternatively, you can set the `LSSupportsOpeningDocumentsInPlace` key to `true` in the `Info.plist` to expose your app's folder in the **Files** app on the iOS device. The destination path to push a file is in the following format: `@:documents/.` [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) [Ruby](https://testingbot.com#) [C#](https://testingbot.com#) ```java // Push an image file driver.pushFile("@com.testingbot.Sample-App:documents/sample.png", new File("/Users/testingbot/Desktop/sample.png")); // Pull file byte[] fileBase64 = driver.pullFile("@com.testingbot.Sample-App:documents/sample.png"); ``` ```javascript // Push file WebdriverIO example const data = Buffer.from("TestingBot").toString('base64'); await driver.pushFile('@com.testingbot.Sample-App:documents/sample.txt', data); // Pull file WebdriverIO example const fileData = await driver.pullFile('@com.testingbot.Sample-App:documents/sample.txt'); ``` ```python # Push an image file driver.push_file('@com.testingbot.Sample-App:documents/sample.png', source_path='/Users/testingbot/Desktop/sample.png') # Push a text file dest_path = '@com.testingbot.Sample-App:documents/sample.txt' driver.push_file(dest_path, 'TestingBot'.encode("utf-8")) # Pull file file_base64 = driver.pull_file(dest_path) ``` ```ruby # Push an image file driver.push_file('@com.testingbot.Sample-App:documents/sample.png', File.read('/Users/testingbot/Desktop/sample.png')) # Pull file driver.pull_file('@com.testingbot.Sample-App:documents/image.png') ``` ```csharp // Push an image file driver.PushFile("@com.testingbot.Sample-App:documents/image.png", new FileInfo("/Users/testingbot/Desktop/sample.png")); // Pull file byte[] fileBase64 = driver.PullFile("@com.testingbot.Sample-App:documents/sample.png"); ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/appium/permission-popups # Handling permission pop-ups While testing your mobile app, it may ask for specific permissions during the test. Various system dialogs or popups may appear, asking the user to grant access. The mobile app might ask the user permission to access the contacts, notifications, media, ... on the device. Below we'll go over the various techniques to handle these events during your mobile testing: - [Allow or Deny all permissions](https://testingbot.com#allPermissions) - [Allow or Deny specific permissions](https://testingbot.com#specificPermissions) ## Allow or Deny all permissions Appium allows you to automatically accept all permission and alert requests. ### Allow or Deny all permissions on Android You can use the `autoGrantPermissions` capability. Appium will inspect the [Android Manifest](https://developer.android.com/guide/topics/manifest/manifest-intro) of the app, extract the required permissions and grant these. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps["appium:autoGrantPermissions"] = true ``` ```java options.setCapability("appium:autoGrantPermissions", true); ``` ```python capabilities = { "appium:autoGrantPermissions": True } ``` ```javascript const capabilities = { "appium:autoGrantPermissions": true } ``` ```csharp options.AddAdditionalAppiumOption("appium:autoGrantPermissions", true); ``` ### Allow or Deny all permissions on iOS On iOS, Appium offers two capabilities which you can use: `autoAcceptAlerts` and `autoDismissAlerts`. `autoAcceptAlerts` will automatically accept (grant) any alerts (including permission popups) that might appear during your test. `autoDismissAlerts` will automatically dismiss (deny) any alerts (including permission popups) that might appear during your test. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps["appium:autoAcceptAlerts"] = true # to accept all alerts caps["appium:autoDismissAlerts"] = true # to dismiss all alerts ``` ```java options.setCapability("appium:autoAcceptAlerts", true); // to accept all alerts options.setCapability("appium:autoDismissAlerts", true); // to dismiss all alerts ``` ```python # Set one of the following capabilities: capabilities["appium:autoAcceptAlerts"] = True # to accept all alerts capabilities["appium:autoDismissAlerts"] = True # to dismiss all alerts ``` ```javascript // Set one of the following capabilities: capabilities["appium:autoAcceptAlerts"] = true; // to accept all alerts capabilities["appium:autoDismissAlerts"] = true; // to dismiss all alerts ``` ```csharp options.AddAdditionalAppiumOption("appium:autoAcceptAlerts", true); // to accept all alerts options.AddAdditionalAppiumOption("appium:autoDismissAlerts", true); // to dismiss all alerts ``` ### Customizing the alert button selector There may be dialogs that appear during your automated tests which have 3 buttons instead of 2, or there's a specific button you want to press instead of the default (allow) button. In this case, you might want to consider using the `acceptAlertButtonSelector` Appium setting, which allows you to pass a selector (a [class-chain expression](https://github.com/facebookarchive/WebDriverAgent/wiki/Class-Chain-Queries-Construction-Rules)) to tell Appium which button to click. For example, you might want to click the 'Allow Once' button in a location permission dialog. To instruct Appium to do this, you can change the `acceptAlertButtonSelector` setting during your test: ```javascript driver.updateSettings({ acceptAlertButtonSelector: "**/XCUIElementTypeButton[`label CONTAINS[c] 'Allow Once'`]" }); ``` Alternatively, you can set this setting at the beginning of your test with a capability: ```javascript const capabilities = { "appium:settings": { "acceptAlertButtonSelector": "**/XCUIElementTypeButton[`label CONTAINS[c] 'Allow Once'`]" } } ``` ## Allow or Deny specific permissions You can also choose to accept or deny alerts and popups during your test. Your test will need to find the element of the alert or popup and perform a `click` action on that element. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby # For Android driver.find_element(:xpath, ".//android.widget.Button[@text='Allow']").click # For iOS driver.find_element(:id, "Allow").click ``` ```java // For Android driver.findElement(By.xpath(".//android.widget.Button[@text='Allow']")).click(); // For iOS driver.findElement(By.id("Allow")).click(); ``` ```python from selenium.webdriver.common.by import By # For Android driver.find_element(By.XPATH, ".//android.widget.Button[@text='Allow']").click() # For iOS driver.find_element(By.ID, "Allow").click() ``` ```javascript // For Android const element = await driver.$(".//android.widget.Button[@text='Allow']"); await element.click(); // For iOS const allowBtn = await driver.$("#Allow"); await allowBtn.click(); ``` ```csharp // For Android IWebElement ll = driver.FindElement(By.XPath(".//android.widget.Button[@text='Allow']")); ll.Click(); // For iOS IWebElement ll = driver.FindElement(By.Id("Allow")); ll.Click(); ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/appium/screen-orientation # Change screen orientation Appium allows you to change the screen orientation of your iOS and Android devices. You can choose to either change the orientation [at the start of the test](https://testingbot.com#caps) or [during a test](https://testingbot.com#during). ## Set screen orientation at the start of your test [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [Node](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby capabilities = { "appium:orientation" => "LANDSCAPE" } ``` ```java options.setCapability("appium:orientation", "LANDSCAPE"); ``` ```php $capabilities = [ "appium:orientation" => "LANDSCAPE" ]; ``` ```python capabilities = { "appium:orientation": "LANDSCAPE" } ``` ```javascript const capabilities = { "appium:orientation": "LANDSCAPE" }; ``` ```csharp options.AddAdditionalAppiumOption("appium:orientation", "LANDSCAPE"); ``` ## Set screen orientation during a test [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [Node](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby @driver.rotation = :landscape ``` ```java ((AppiumDriver) driver).rotate(ScreenOrientation.LANDSCAPE); ``` ```php // php-webdriver has no rotate() helper; use the Appium W3C "mobile:" command $driver->executeScript('mobile: setOrientation', ['orientation' => 'LANDSCAPE']); ``` ```python driver.orientation = "LANDSCAPE" ``` ```javascript await driver.setOrientation('landscape'); ``` ```csharp driver.Orientation = ScreenOrientation.Landscape; ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/appium/localization-options # Localization testing on iOS and Android When running tests with Appium, you can choose to set a specific locale for the device or emulator, by using the `appium:locale` capability. Specifying this capability will make sure the device/emulator and app will display or format data such as dates, time, decimal separators and calendars according to the specified country's regional conventions. **The expected value format depends on the platform:** - **Android:** two-letter region code, e.g. `ES` for Spain (see the Android table below). The example below uses `"ES"`. - **iOS:** language\_region locale identifier, e.g. `es_ES`, `fr_CA` (see the iOS table below). On iOS, the example below would be `"es_ES"` instead of `"ES"`. You can also change the Simulator UI locale at runtime using the script command `mobile: configureLocalization`. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby capabilities = { "appium:locale" => "ES" } ``` ```java options.setCapability("appium:locale", "ES"); ``` ```php $capabilities = [ "appium:locale" => "ES" ]; ``` ```python capabilities = { "appium:locale": "ES" } ``` ```javascript const capabilities = { "appium:locale": "ES" }; ``` ```csharp options.AddAdditionalAppiumOption("appium:locale", "ES"); ``` ## Android locale codes | Locale | Locale Code | | --- | --- | | Australia | AU | | Austria | AT | | Belgium | BE | | Brazil | BR | | Britain | GB | | Bulgaria | BG | | Canada | CA | | Croatia | HR | | Czech Republic | CZ | | Denmark | DK | | Egypt | EG | | Finland | FI | | France | FR | | Germany | DE | | Greece | GR | | Hong-Kong | HK | | Hungary | HU | | India | IN | | Indonesia | ID | | Ireland | IE | | Israel | IL | | Italy | IT | | Japan | JP | | Korea | KR | | Latvia | LV | | Liechtenstein | LI | | Lithuania | LT | | Mexico | MX | | Netherlands | NL | | New Zealand | NZ | | Norway | NO | | Philippines | PH | | Poland | PL | | Portugal | PT | | PRC | CN | | Romania | RO | | Russia | RU | | Serbia | RS | | Singapore | SG | | Slovakia | SK | | Slovenia | SI | | Spain | ES | | Sweden | SE | | Switzerland | CH | | Taiwan | TW | | Thailand | TH | | Turkey | TR | | Ukraine | UA | | US | US | | USA | US | | Vietnam | VN | | Zimbabwe | ZW | ## iOS locale codes | Locale | Locale Code | | --- | --- | | Australia  | en\_AU | | Belgium  | nl\_BE | | Belgium  | fr\_BE | | Brunei Darussalam | ms\_BN | | Canada  | en\_CA | | Canada | fr\_CA | | Czech Republic  | cs\_CZ | | Finland | fi\_FI | | Germany  | de\_DE | | Greece | el\_GR | | Hungary | hu\_HU | | India | hi\_IN | | Indonesia | id\_ID | | Israel | he\_IL | | Italy | it\_IT | | Japan | ja\_JP | | Malaysia | ms\_MY | | Netherlands  | nl\_NL | | New Zealand   | en\_NZ | | Norway | nb\_NO | | Philippines  | tl\_PH | | Poland | pl\_PL | | PRC  | zh\_CN | | Romania | ro\_RO | | Russia | ru\_RU | | Singapore | en\_SG | | Slovakia | sk\_SK | | Korea | ko\_KR | | Sweden  | sv\_SE | | Taiwan  | zh\_TW | | Thailand  | th\_TH | | Turkey  | tr\_TR | | UK  | en\_GB | | Ukraine  | uk\_UA | | US | es\_US | | USA | en\_US | | Vietnam  | vi\_VN | | Brazil | pt-BR | | China (Simplified) | zh-Hans | | China (Traditional) | zh-Hant | | Hong Kong | zh-HK | | India | en-IN | | Ireland | en-IE | | Latin America | es-419 | | Mexico | es-MX | | South Africa | en-ZA | ## Change Language Specify the `appium:language` capability to set the language of the app under test, and the device/emulator. On iOS, you can set the language to for example French: `fr`. More information is available in the [Language IDs documentation](https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPInternational/LanguageandLocaleIDs/LanguageandLocaleIDs.html). You can also change the Simulator UI language at runtime using the following script command: `mobile: configureLocalization`. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby capabilities = { "appium:language" => "es" } ``` ```java options.setCapability("appium:language", "es"); ``` ```php $capabilities = [ "appium:language" => "es" ]; ``` ```python capabilities = { "appium:language": "es" } ``` ```javascript const capabilities = { "appium:language": "es" }; ``` ```csharp options.AddAdditionalAppiumOption("appium:language", "es"); ``` ### Android language codes | Language | Language Code | | --- | --- | | Arabic | ar | | Bulgarian | bg | | Catalan | ca | | Chinese | zh | | Croatian | hr | | Czech | cs | | Danish | da | | Dutch | nl | | English | en | | Finnish | fi | | French | fr | | German | de | | Greek | el | | Hebrew | he | | Hindi | hi | | Hungarian | hu | | Indonesian | id | | Italian | it | | Japanese | ja | | Korean | ko | | Latvian | lv | | Lithuanian | lt | | Norwegian-Bokmol | nb | | Polish | pl | | Portuguese | pt | | Romanian | ro | | Russian | ru | | Serbian | sr | | Slovak | sk | | Slovenian | sl | | Spanish | es | | Swedish | sv | | Tagalog | tl | | Thai | th | | Turkish | tr | | Ukrainian | uk | | Vietnamese | vi | ### iOS language codes | Language | Language Code | | --- | --- | | Chinese | zh | | Czech | cs | | Dutch | nl | | English | en | | Finnish | fi | | French | fr | | German | de | | Greek | el | | Hebrew | he | | Hindi | hi | | Hungarian | hu | | Indonesian | id | | Italian | it | | Japanese | ja | | Korean | ko | | Malay | ms | | Norwegian (Bokmal) | nb | | Polish | pl | | Portuguese (Brazil) | pt | | Romanian | ro | | Russian | ru | | Slovak | sk | | Spanish | es | | Swedish | sv | | Tagalog | tl | | Thai | th | | Turkish | tr | | Ukrainian | uk | | Vietnamese | vi | ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/appium/dark-mode # Dark Mode testing with Appium Dark mode is a setting, available on both Android and iOS, which toggles the interface appearance of the phone to use either the standard light mode setting or a dark mode. When dark mode is enabled, the phone will switch to a dark background with lighter foreground colors, which enables better viewing in low-light conditions. ## Changing Dark Mode This example will use `mobile: setAppearance` to set the appearance to dark (dark mode). [Android](https://testingbot.com#) [iOS](https://testingbot.com#) ```javascript const { remote } = require('webdriverio'); const capabilities = { 'platformName': 'Android', 'appium:app': 'https://testingbot.com/appium/sample.apk', 'appium:deviceName': 'Pixel 8', 'appium:platformVersion': '14.0', 'appium:automationName': 'UiAutomator2', 'tb:options': { 'name': 'Dark Mode Test', 'realDevice': true } }; (async () => { const driver = await remote({ hostname: 'hub.testingbot.com', port: 443, protocol: 'https', path: '/wd/hub', user: 'api_key', key: 'api_secret', capabilities }); // Enable dark mode await driver.execute('mobile: setAppearance', { style: 'dark' }); const inputA = await driver.$('~inputA'); await inputA.setValue('10'); const inputB = await driver.$('~inputB'); await inputB.setValue('5'); const sum = await driver.$('~sum'); const value = await sum.getText(); console.assert(value === '15', 'Sum should be 15'); await driver.deleteSession(); })(); ``` Make sure to always stop your test (`driver.deleteSession()`), otherwise it will continue running, leading to a timeout. ```javascript const { remote } = require('webdriverio'); const capabilities = { 'platformName': 'iOS', 'appium:app': 'https://testingbot.com/appium/sample.ipa', 'appium:deviceName': 'iPhone 15', 'appium:platformVersion': '17.0', 'appium:automationName': 'XCUITest', 'tb:options': { 'name': 'Dark Mode Test', 'realDevice': true } }; (async () => { const driver = await remote({ hostname: 'hub.testingbot.com', port: 443, protocol: 'https', path: '/wd/hub', user: 'api_key', key: 'api_secret', capabilities }); // Enable dark mode await driver.execute('mobile: setAppearance', { style: 'dark' }); const inputA = await driver.$('~inputA'); await inputA.setValue('10'); const inputB = await driver.$('~inputB'); await inputB.setValue('5'); await driver.deleteSession(); })(); ``` Make sure to always stop your test (`driver.deleteSession()`), otherwise it will continue running, leading to a timeout. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/appium/push-notifications # Testing Push Notifications with Appium Push notifications are a critical part of many mobile applications. With TestingBot, you can simulate and interact with push notifications on both Android and iOS devices during your Appium tests. **Prerequisite:** Your app must request push notification permission and the user must allow it before notifications can be displayed. To handle this automatically during tests, set the `autoGrantPermissions` capability (Android) or `autoAcceptAlerts` capability (iOS). If your app has not registered for notifications, none of the approaches below will show a visible notification. ```java // Android: auto-grant all permissions including notifications options.setCapability("appium:autoGrantPermissions", true); // iOS: auto-accept permission dialogs (e.g. "Allow Notifications") options.setCapability("appium:autoAcceptAlerts", true); ``` ## Which method should I use? - **[Android](https://testingbot.com#android)**, Trigger real push notifications via FCM or use ADB shell commands via Appium's `mobile: shell` on emulators and real devices. - **[iOS Simulators](https://testingbot.com#ios-simulator)**, Use Appium's `mobile: pushNotification` command to send simulated notifications with a JSON payload. - **[iOS Real Devices](https://testingbot.com#ios-real)**, Trigger push notifications via your backend server or a third-party service (APNs, Firebase) to the device under test. - **[Interact with Notifications](https://testingbot.com#interact)**, Open the notification shade or banner and tap on notifications during your test. ## Android Push Notifications There are two approaches to test push notifications on Android with TestingBot: - **Trigger via your backend (recommended):** Send a real push notification through FCM (Firebase Cloud Messaging) to the app running on the device. The notification appears with the correct app icon and package attribution, exactly as your end users would see it. - **Simulate via ADB shell:** Use Appium's `mobile: shell` command to post a system notification. This is a quick way to test notification interaction (tapping, swiping), but the notification appears as a generic system notification without your app's icon. ### Option 1: Trigger a real notification via FCM (recommended) If your app uses Firebase Cloud Messaging, you can send a test push notification by calling the FCM HTTP v1 API during your test. The notification will be delivered to your app on the TestingBot device, showing the correct app icon and notification channel. You'll need a **Firebase service account key** (JSON file) for your project and the **FCM registration token** from the device. Most apps log the registration token on startup, you can retrieve it from logcat or have your app expose it via a test endpoint. Your app must already be set up to receive FCM notifications. This approach sends a real notification through your existing push infrastructure. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [C#](https://testingbot.com#) ```java import io.appium.java_client.android.AndroidDriver; import io.appium.java_client.android.options.UiAutomator2Options; import java.net.URL; import java.net.URI; import java.net.http.HttpClient; import java.net.http.HttpRequest; import java.net.http.HttpResponse; import java.util.HashMap; import java.util.Map; public class PushNotificationTest { public static void main(String[] args) throws Exception { UiAutomator2Options options = new UiAutomator2Options(); options.setDeviceName("Pixel 8"); options.setPlatformVersion("14.0"); options.setApp("tb://your_app_id"); Map tbOptions = new HashMap<>(); tbOptions.put("key", "api_key"); tbOptions.put("secret", "api_secret"); tbOptions.put("name", "Push Notification Test"); tbOptions.put("realDevice", true); options.setCapability("tb:options", tbOptions); AndroidDriver driver = new AndroidDriver( new URL("https://hub.testingbot.com/wd/hub"), options); // Retrieve the FCM registration token from your app // (e.g. via a test endpoint or logcat) String fcmToken = "DEVICE_FCM_REGISTRATION_TOKEN"; // Send a real push notification via FCM HTTP v1 API String projectId = "your-firebase-project-id"; String accessToken = "YOUR_OAUTH2_ACCESS_TOKEN"; String json = "{" + "\"message\": {" + " \"token\": \"" + fcmToken + "\"," + " \"notification\": {" + " \"title\": \"Order Update\"," + " \"body\": \"Your order has been shipped!\"" + " }" + "}" + "}"; HttpRequest request = HttpRequest.newBuilder() .uri(URI.create("https://fcm.googleapis.com/v1/projects/" + projectId + "/messages:send")) .header("Authorization", "Bearer " + accessToken) .header("Content-Type", "application/json") .POST(HttpRequest.BodyPublishers.ofString(json)) .build(); HttpClient.newHttpClient().send(request, HttpResponse.BodyHandlers.ofString()); Thread.sleep(3000); driver.quit(); } } ``` ```python from appium import webdriver from appium.options.android import UiAutomator2Options import requests import time options = UiAutomator2Options() options.device_name = 'Pixel 8' options.platform_version = '14.0' options.app = 'tb://your_app_id' options.set_capability('tb:options', { 'key': 'api_key', 'secret': 'api_secret', 'name': 'Push Notification Test', 'realDevice': True }) driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options) # Retrieve the FCM registration token from your app fcm_token = 'DEVICE_FCM_REGISTRATION_TOKEN' # Send a real push notification via FCM HTTP v1 API project_id = 'your-firebase-project-id' access_token = 'YOUR_OAUTH2_ACCESS_TOKEN' response = requests.post( f'https://fcm.googleapis.com/v1/projects/{project_id}/messages:send', headers={ 'Authorization': f'Bearer {access_token}', 'Content-Type': 'application/json' }, json={ 'message': { 'token': fcm_token, 'notification': { 'title': 'Order Update', 'body': 'Your order has been shipped!' } } } ) time.sleep(3) driver.quit() ``` ```javascript const { remote } = require('webdriverio'); const capabilities = { 'platformName': 'Android', 'appium:app': 'tb://your_app_id', 'appium:deviceName': 'Pixel 8', 'appium:platformVersion': '14.0', 'appium:automationName': 'UiAutomator2', 'tb:options': { 'name': 'Push Notification Test', 'realDevice': true } }; const driver = await remote({ hostname: 'hub.testingbot.com', port: 443, protocol: 'https', path: '/wd/hub', user: 'api_key', key: 'api_secret', capabilities }); // Retrieve the FCM registration token from your app const fcmToken = 'DEVICE_FCM_REGISTRATION_TOKEN'; // Send a real push notification via FCM HTTP v1 API const projectId = 'your-firebase-project-id'; const accessToken = 'YOUR_OAUTH2_ACCESS_TOKEN'; await fetch( `https://fcm.googleapis.com/v1/projects/${projectId}/messages:send`, { method: 'POST', headers: { 'Authorization': `Bearer ${accessToken}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ message: { token: fcmToken, notification: { title: 'Order Update', body: 'Your order has been shipped!' } } }) } ); await driver.pause(3000); await driver.deleteSession(); ``` ```ruby require 'appium_lib' require 'net/http' require 'json' caps = { 'platformName' => 'Android', 'appium:app' => 'tb://your_app_id', 'appium:deviceName' => 'Pixel 8', 'appium:platformVersion' => '14.0', 'appium:automationName' => 'UiAutomator2', 'tb:options' => { 'key' => 'api_key', 'secret' => 'api_secret', 'name' => 'Push Notification Test', 'realDevice' => true } } driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: 'https://hub.testingbot.com/wd/hub' } }).start_driver # Retrieve the FCM registration token from your app fcm_token = 'DEVICE_FCM_REGISTRATION_TOKEN' # Send a real push notification via FCM HTTP v1 API project_id = 'your-firebase-project-id' access_token = 'YOUR_OAUTH2_ACCESS_TOKEN' uri = URI("https://fcm.googleapis.com/v1/projects/#{project_id}/messages:send") http = Net::HTTP.new(uri.host, uri.port) http.use_ssl = true request = Net::HTTP::Post.new(uri) request['Authorization'] = "Bearer #{access_token}" request['Content-Type'] = 'application/json' request.body = { message: { token: fcm_token, notification: { title: 'Order Update', body: 'Your order has been shipped!' } } }.to_json http.request(request) sleep 3 driver.quit ``` ```csharp using OpenQA.Selenium.Appium; using OpenQA.Selenium.Appium.Android; using System.Text; using System.Text.Json; var options = new AppiumOptions(); options.PlatformName = "Android"; options.AddAdditionalAppiumOption("appium:app", "tb://your_app_id"); options.AddAdditionalAppiumOption("appium:deviceName", "Pixel 8"); options.AddAdditionalAppiumOption("appium:platformVersion", "14.0"); options.AddAdditionalAppiumOption("appium:automationName", "UiAutomator2"); var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["name"] = "Push Notification Test", ["realDevice"] = true }; options.AddAdditionalAppiumOption("tb:options", tbOptions); var driver = new AndroidDriver( new Uri("https://hub.testingbot.com/wd/hub"), options); // Retrieve the FCM registration token from your app var fcmToken = "DEVICE_FCM_REGISTRATION_TOKEN"; // Send a real push notification via FCM HTTP v1 API var projectId = "your-firebase-project-id"; var accessToken = "YOUR_OAUTH2_ACCESS_TOKEN"; var payload = JsonSerializer.Serialize(new { message = new { token = fcmToken, notification = new { title = "Order Update", body = "Your order has been shipped!" } } }); using var client = new HttpClient(); client.DefaultRequestHeaders.Add("Authorization", $"Bearer {accessToken}"); var content = new StringContent(payload, Encoding.UTF8, "application/json"); await client.PostAsync( $"https://fcm.googleapis.com/v1/projects/{projectId}/messages:send", content); Thread.Sleep(3000); driver.Quit(); ``` ### Option 2: Quick test with ADB shell For quick testing of notification _interaction_ (tapping, swiping away), you can post a system notification via ADB. This uses Appium's `mobile: shell` command. Notifications created with `cmd notification post` are generic system notifications. They will **not** display your app's icon or appear as coming from your app. Use the [FCM approach](https://testingbot.com#android-fcm) above if you need app-branded notifications. ```java // Post a generic system notification Map shellArgs = new HashMap<>(); shellArgs.put("command", "cmd"); shellArgs.put("args", new String[]{ "notification", "post", "-S", "messaging", "-t", "Test Title", "Test Message"}); driver.executeScript("mobile: shell", shellArgs); // Post a notification targeting a specific package (shows app icon on some Android versions) Map packageArgs = new HashMap<>(); packageArgs.put("command", "cmd"); packageArgs.put("args", new String[]{ "notification", "post", "-S", "messaging", "--package", "com.example.myapp", "-t", "Order Update", "Your order has been shipped!"}); driver.executeScript("mobile: shell", packageArgs); ``` `mobile: shell` is a security-gated feature in Appium 2 and 3: the Appium server must be launched with the insecure feature enabled (`--allow-insecure uiautomator2:adb_shell`). If this feature is not enabled on the TestingBot grid for your session, the call fails with a "not enabled" error. Contact support to confirm availability, or prefer the FCM approach above for cloud runs. You can also set the `adbExecTimeout` capability if a command runs longer than the default timeout. ## iOS Simulator Notifications On iOS simulators, Appium provides the `mobile: pushNotification` command that accepts a JSON payload matching the Apple Push Notification format. This allows you to test how your app handles push notifications without needing a real APNs connection. The `mobile: pushNotification` command only works on **iOS simulators** , not on real iOS devices. For real device testing, see the [iOS Real Devices](https://testingbot.com#ios-real) section. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [C#](https://testingbot.com#) ```java import io.appium.java_client.ios.IOSDriver; import io.appium.java_client.ios.options.XCUITestOptions; import java.net.URL; import java.util.HashMap; import java.util.Map; public class IOSPushNotificationTest { public static void main(String[] args) throws Exception { XCUITestOptions options = new XCUITestOptions(); options.setDeviceName("iPhone 16"); options.setPlatformVersion("18.0"); options.setApp("tb://your_app_id"); Map tbOptions = new HashMap<>(); tbOptions.put("key", "api_key"); tbOptions.put("secret", "api_secret"); tbOptions.put("name", "iOS Push Notification Test"); options.setCapability("tb:options", tbOptions); IOSDriver driver = new IOSDriver( new URL("https://hub.testingbot.com/wd/hub"), options); // Simulate a push notification Map payload = new HashMap<>(); payload.put("bundleId", "com.example.myapp"); Map alert = new HashMap<>(); alert.put("title", "New Message"); alert.put("body", "You have a new message from TestingBot"); alert.put("subtitle", "Testing"); Map aps = new HashMap<>(); aps.put("alert", alert); aps.put("badge", 1); aps.put("sound", "default"); Map payloadData = new HashMap<>(); payloadData.put("aps", aps); payload.put("payload", payloadData); driver.executeScript("mobile: pushNotification", payload); Thread.sleep(2000); driver.quit(); } } ``` ```python from appium import webdriver from appium.options.ios import XCUITestOptions import time options = XCUITestOptions() options.device_name = 'iPhone 16' options.platform_version = '18.0' options.app = 'tb://your_app_id' options.set_capability('tb:options', { 'key': 'api_key', 'secret': 'api_secret', 'name': 'iOS Push Notification Test' }) driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options) # Simulate a push notification driver.execute_script('mobile: pushNotification', { 'bundleId': 'com.example.myapp', 'payload': { 'aps': { 'alert': { 'title': 'New Message', 'body': 'You have a new message from TestingBot', 'subtitle': 'Testing' }, 'badge': 1, 'sound': 'default' } } }) time.sleep(2) driver.quit() ``` ```javascript const { remote } = require('webdriverio'); const capabilities = { 'platformName': 'iOS', 'appium:app': 'tb://your_app_id', 'appium:deviceName': 'iPhone 16', 'appium:platformVersion': '18.0', 'appium:automationName': 'XCUITest', 'tb:options': { 'name': 'iOS Push Notification Test' } }; const driver = await remote({ hostname: 'hub.testingbot.com', port: 443, protocol: 'https', path: '/wd/hub', user: 'api_key', key: 'api_secret', capabilities }); // Simulate a push notification await driver.execute('mobile: pushNotification', { bundleId: 'com.example.myapp', payload: { aps: { alert: { title: 'New Message', body: 'You have a new message from TestingBot', subtitle: 'Testing' }, badge: 1, sound: 'default' } } }); await driver.pause(2000); await driver.deleteSession(); ``` ```ruby require 'appium_lib' caps = { 'platformName' => 'iOS', 'appium:app' => 'tb://your_app_id', 'appium:deviceName' => 'iPhone 16', 'appium:platformVersion' => '18.0', 'appium:automationName' => 'XCUITest', 'tb:options' => { 'key' => 'api_key', 'secret' => 'api_secret', 'name' => 'iOS Push Notification Test' } } driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: 'https://hub.testingbot.com/wd/hub' } }).start_driver # Simulate a push notification driver.execute_script('mobile: pushNotification', { bundleId: 'com.example.myapp', payload: { aps: { alert: { title: 'New Message', body: 'You have a new message from TestingBot', subtitle: 'Testing' }, badge: 1, sound: 'default' } } }) sleep 2 driver.quit ``` ```csharp using OpenQA.Selenium.Appium; using OpenQA.Selenium.Appium.iOS; var options = new AppiumOptions(); options.PlatformName = "iOS"; options.AddAdditionalAppiumOption("appium:app", "tb://your_app_id"); options.AddAdditionalAppiumOption("appium:deviceName", "iPhone 16"); options.AddAdditionalAppiumOption("appium:platformVersion", "18.0"); options.AddAdditionalAppiumOption("appium:automationName", "XCUITest"); var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["name"] = "iOS Push Notification Test" }; options.AddAdditionalAppiumOption("tb:options", tbOptions); var driver = new IOSDriver( new Uri("https://hub.testingbot.com/wd/hub"), options); // Simulate a push notification var payload = new Dictionary { ["bundleId"] = "com.example.myapp", ["payload"] = new Dictionary { ["aps"] = new Dictionary { ["alert"] = new Dictionary { ["title"] = "New Message", ["body"] = "You have a new message from TestingBot", ["subtitle"] = "Testing" }, ["badge"] = 1, ["sound"] = "default" } } }; driver.ExecuteScript("mobile: pushNotification", payload); Thread.Sleep(2000); driver.Quit(); ``` ### Payload format The `mobile: pushNotification` command accepts a payload that follows the [Apple Push Notification format](https://developer.apple.com/documentation/usernotifications/generating-a-remote-notification). The key fields are: - **`bundleId`:** The bundle identifier of the target app (e.g. `com.example.myapp`). - **`payload.aps.alert.title`:** The title text shown in the notification banner. - **`payload.aps.alert.body`:** The main message text of the notification. - **`payload.aps.badge`:** The badge number to display on the app icon. - **`payload.aps.sound`:** The notification sound. Use `"default"` for the standard notification sound. You can also include custom key-value pairs in the payload for your app to process: ```java Map alert = new HashMap<>(); alert.put("title", "Order Update"); alert.put("body", "Your order #1234 has been shipped"); Map aps = new HashMap<>(); aps.put("alert", alert); Map payload = new HashMap<>(); payload.put("aps", aps); payload.put("orderId", "1234"); payload.put("deepLink", "/orders/1234"); Map pushArgs = new HashMap<>(); pushArgs.put("bundleId", "com.example.myapp"); pushArgs.put("payload", payload); driver.executeScript("mobile: pushNotification", pushArgs); ``` ### Wait for a notification with `mobile: expectNotification` The XCUITest driver also provides [`mobile: expectNotification`](https://appium.github.io/appium-xcuitest-driver/latest/execute-methods/#mobile-expectnotification), which blocks test execution until a specific notification is delivered. This is useful when you want to verify that your app posts a particular notification in response to an action, without polling for UI elements. - **`name` (required):** The notification name to wait for (e.g. `com.example.orderComplete`). - **`type`:** `"plain"` (default) for standard NSNotificationCenter notifications, or `"darwin"` for system-level Darwin notifications. - **`timeoutSeconds`:** Maximum time to wait in seconds. Defaults to 60. Throws a timeout error if the notification is not received. ```java // Wait up to 10 seconds for a specific notification Map expectArgs = new HashMap<>(); expectArgs.put("name", "com.example.myapp.orderComplete"); expectArgs.put("timeoutSeconds", 10); driver.executeScript("mobile: expectNotification", expectArgs); ``` `mobile: expectNotification` listens for _internal_ notifications (NSNotificationCenter / Darwin), not push notification banners. Use it to verify that your app's code reacts correctly to events, alongside `mobile: pushNotification` which triggers the visible notification. ## iOS Real Device Notifications Appium's `mobile: pushNotification` command does not work on real iOS devices because Apple only allows push notifications through APNs (Apple Push Notification service). To test push notifications on real iOS devices, you need to trigger them from your backend or via FCM/APNs. ### Disable app re-signing When you upload an iOS app to TestingBot, it is automatically [re-signed with a TestingBot provisioning profile](https://testingbot.com/support/app-automate/help/app-resigning) so it can be installed on our devices. This re-signing **removes your app's push notification entitlements** , which means APNs will reject push tokens from the re-signed app and notifications will not be delivered. To test push notifications on real iOS devices, you must **disable re-signing** by setting `resigningEnabled` to `false`. This requires your app to be signed with an [Apple Developer Enterprise certificate](https://developer.apple.com/programs/enterprise/), which allows installation on any device without TestingBot's provisioning profile. ```java // Disable re-signing so push notification entitlements are preserved Map tbOptions = new HashMap<>(); tbOptions.put("key", "api_key"); tbOptions.put("secret", "api_secret"); tbOptions.put("resigningEnabled", false); options.setCapability("tb:options", tbOptions); ``` Without `"resigningEnabled": false`, your app's push entitlements will be stripped during re-signing and push notifications will silently fail on real iOS devices. ### Trigger the notification Once re-signing is disabled and your app retains its push entitlements, trigger notifications from your backend: 1. **Use your app's backend API** , If your app has an endpoint that triggers push notifications, call it from your test to send a real notification to the device under test. 2. **Use Firebase Cloud Messaging (FCM)**, Send a push notification via the FCM API. Your app must be configured to receive FCM messages. 3. **Use APNs directly** , Send a notification via the Apple Push Notification service using a valid APNs certificate or key. ### Example: Trigger via your backend API [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) ```java import java.net.URI; import java.net.http.HttpClient; import java.net.http.HttpRequest; import java.net.http.HttpResponse; // After your app is running and the device token is registered, // trigger a notification from your backend String json = "{\"device_token\": \"DEVICE_TOKEN_FROM_APP\"," + "\"title\": \"New Message\"," + "\"body\": \"You have a new message\"}"; HttpRequest request = HttpRequest.newBuilder() .uri(URI.create("https://your-api.example.com/send-notification")) .header("Content-Type", "application/json") .POST(HttpRequest.BodyPublishers.ofString(json)) .build(); HttpClient.newHttpClient().send(request, HttpResponse.BodyHandlers.ofString()); // Wait for the notification to arrive Thread.sleep(3000); // Now interact with the notification in your Appium test // (see the "Interact with Notifications" section below) ``` ```python import requests import time # After your app is running and the device token is registered, # trigger a notification from your backend requests.post('https://your-api.example.com/send-notification', json={ 'device_token': 'DEVICE_TOKEN_FROM_APP', 'title': 'New Message', 'body': 'You have a new message' }) # Wait for the notification to arrive time.sleep(3) # Now interact with the notification in your Appium test # (see the "Interact with Notifications" section below) ``` ```javascript // After your app is running and the device token is registered, // trigger a notification from your backend await fetch('https://your-api.example.com/send-notification', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ device_token: 'DEVICE_TOKEN_FROM_APP', title: 'New Message', body: 'You have a new message' }) }); // Wait for the notification to arrive await driver.pause(3000); // Now interact with the notification in your Appium test ``` ```ruby require 'net/http' require 'json' # After your app is running and the device token is registered, # trigger a notification from your backend uri = URI('https://your-api.example.com/send-notification') Net::HTTP.post(uri, { device_token: 'DEVICE_TOKEN_FROM_APP', title: 'New Message', body: 'You have a new message' }.to_json, 'Content-Type' => 'application/json') # Wait for the notification to arrive sleep 3 # Now interact with the notification in your Appium test ``` ## Interact with Notifications After a notification is displayed, you can interact with it in your Appium test. The approach differs between Android and iOS. ### Android: Open the notification shade On Android, use `openNotifications()` to pull down the notification shade, then find and tap the notification element. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [C#](https://testingbot.com#) ```java // Open the notification shade driver.openNotifications(); // Wait for the notification to appear WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(10)); WebElement notification = wait.until(ExpectedConditions .presenceOfElementLocated(By.xpath( "//*[contains(@text, 'Test Title')]"))); // Tap the notification notification.click(); ``` ```python from appium.webdriver.common.appiumby import AppiumBy from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC # Open the notification shade driver.open_notifications() # Wait for the notification to appear wait = WebDriverWait(driver, 10) notification = wait.until(EC.presence_of_element_located(( AppiumBy.XPATH, "//*[contains(@text, 'Test Title')]" ))) # Tap the notification notification.click() ``` ```javascript // Open the notification shade (WebdriverIO) await driver.execute('mobile: openNotifications'); // Wait for the notification to appear const notification = await driver.$('//*[contains(@text, "Test Title")]'); await notification.waitForExist({ timeout: 10000 }); // Tap the notification await notification.click(); ``` ```ruby # Open the notification shade driver.open_notifications # Wait for the notification to appear wait = Selenium::WebDriver::Wait.new(timeout: 10) notification = wait.until { driver.find_element(:xpath, "//*[contains(@text, 'Test Title')]") } # Tap the notification notification.click ``` ```csharp // Open the notification shade driver.OpenNotifications(); // Wait for the notification to appear var wait = new WebDriverWait(driver, TimeSpan.FromSeconds(10)); var notification = wait.Until(d => d.FindElement(By.XPath("//*[contains(@text, 'Test Title')]"))); // Tap the notification notification.Click(); ``` ### iOS: Tap the notification banner On iOS, notifications appear as banners at the top of the screen. You can interact with them using the Notification Center or by tapping the banner directly. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) ```java // Wait for the notification banner to appear WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(10)); WebElement banner = wait.until(ExpectedConditions .presenceOfElementLocated(By.name("NotificationShortLookView"))); // Tap the banner to open the notification banner.click(); ``` ```python from appium.webdriver.common.appiumby import AppiumBy from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC # Wait for the notification banner to appear wait = WebDriverWait(driver, 10) banner = wait.until(EC.presence_of_element_located(( AppiumBy.ACCESSIBILITY_ID, 'NotificationShortLookView' ))) # Tap the banner to open the notification banner.click() ``` ```javascript // Wait for the notification banner to appear const banner = await driver.$('~NotificationShortLookView'); await banner.waitForExist({ timeout: 10000 }); // Tap the banner to open the notification await banner.click(); ``` ```ruby # Wait for the notification banner to appear wait = Selenium::WebDriver::Wait.new(timeout: 10) banner = wait.until { driver.find_element(:accessibility_id, 'NotificationShortLookView') } # Tap the banner to open the notification banner.click ``` ## Tips & Best Practices ### Grant notification permissions Your app needs notification permissions before it can display push notifications. Use the `autoGrantPermissions` capability (Android) or `autoAcceptAlerts` capability (iOS) to automatically handle permission dialogs. See the [Permission Popups](https://testingbot.com/support/app-automate/appium/permission-popups) documentation for more details. ```java // Android: auto-grant all permissions options.setCapability("appium:autoGrantPermissions", true); // iOS: auto-accept permission dialogs options.setCapability("appium:autoAcceptAlerts", true); ``` ### Wait for notifications to arrive Push notifications are asynchronous. Always use explicit waits instead of fixed delays when checking for notifications. Use `WebDriverWait` to poll for the notification element rather than `sleep()`. On iOS simulators, you can also use [`mobile: expectNotification`](https://testingbot.com#ios-expect) to block until an internal notification is delivered. ### Use unique notification content When searching for a notification in the notification shade, use unique text content (such as a timestamp or test ID) to avoid matching stale or system notifications. ### Dismiss notifications after testing After interacting with a notification, dismiss the notification shade (Android) or return to the app to continue your test flow. ```java // Android: press Back to close the notification shade ((AndroidDriver) driver).pressKey(new KeyEvent(AndroidKey.BACK)); // iOS: swipe up on the notification banner to dismiss Map swipeArgs = new HashMap<>(); swipeArgs.put("direction", "up"); driver.executeScript("mobile: swipe", swipeArgs); ``` ## Frequently Asked Questions ### Can I test push notifications on real iOS devices? Yes, but you cannot simulate them using Appium's `mobile: pushNotification` command, that only works on iOS simulators. For real iOS devices, you need to trigger actual push notifications through your backend, Firebase Cloud Messaging (FCM), or APNs (Apple Push Notification service). Your test can then interact with the notification banner or Notification Center using Appium's standard element interaction commands. See the [iOS Real Devices](https://testingbot.com#ios-real) section for examples. ### How do I simulate a push notification on Android with Appium? On Android, use Appium's `mobile: shell` command with the ADB `cmd notification post` command. This sends a notification to the Android notification shade. Example: `driver.execute_script('mobile: shell', {'command': 'cmd', 'args': ['notification', 'post', '-S', 'messaging', '-t', 'Title', 'Message']})`. This works on both emulators and real Android devices on TestingBot. ### How do I open the notification shade in an Appium test? On Android, use `driver.openNotifications()` (Java/Python/Ruby/C#) or `driver.execute('mobile: openNotifications')` (WebdriverIO). This pulls down the system notification shade. On iOS, notifications appear as banners at the top of the screen, wait for the banner element and tap it directly using its accessibility identifier. ### Why is my push notification not appearing during the test? Common causes: 1. The app has not been granted notification permissions, use `autoGrantPermissions` (Android) or `autoAcceptAlerts` (iOS) to handle permission dialogs. 2. On iOS real devices, `mobile: pushNotification` only works on simulators, use your backend or FCM/APNs instead. 3. The notification may take a few seconds to appear, use explicit waits instead of fixed sleeps. See the [Tips](https://testingbot.com#tips) section for more details. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/appium/options # Appium tb:options Reference Reference for the capabilities you can pass to Appium tests on TestingBot: pick a device, pin an Appium version, change timezone or geolocation, throttle the network, record video, group tests under a build, attach custom metadata, and more. Each section includes code examples in Java, Node.js, Python, Ruby and C#. The [Appium Capabilities generator](https://testingbot.com/support/app-automate/appium/capabilities) allows you to easily generate the necessary capabilities for your mobile tests. ## Appium Options Appium provides [a lot of options](https://appium.io/docs/en/2.0/guides/caps/) to configure your test. Some important options that might help: **For Android:** - **appActivity** and **appPackage** : by default, Appium will try to extract the main Activity from your apk. If this fails, you can supply your own with these options. - **chromeOptions** : additional chromedriver options you can supply. - **otherApps** : a JSON array of other apps that need to be installed, in URL format. **For Android & iOS:** - **locale** : the language of the simulator/device (ex: fr\_CA) - **newCommandTimeout** : How long (in seconds) Appium will wait for a new command from the client before assuming the client quit and ending the session ## Specifying Appium Version TestingBot will use the most recent, compatible, Appium version according to the device, OS and version you specify. If you'd like to specify your own Appium version, you can do this with the `appiumVersion` capability. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { "platformName" => "iOS", "appium:automationName" => "XCUITest", "appium:deviceName" => "iPhone 13", "appium:platformVersion" => "16.0", "appium:app" => "https://testingbot.com/appium/sample.ipa", "tb:options" => { "appiumVersion" => "2.0" } } driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "https://api_key:api_secret@hub.testingbot.com/wd/hub" } }, true).start_driver ``` ```java XCUITestOptions options = new XCUITestOptions(); options.setPlatformName("iOS"); options.setAutomationName("XCUITest"); options.setDeviceName("iPhone 13"); options.setPlatformVersion("16.0"); options.setApp("https://testingbot.com/appium/sample.ipa"); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("appiumVersion", "2.0"); options.setCapability("tb:options", tbOptions); IOSDriver driver = new IOSDriver( new URL("https://hub.testingbot.com/wd/hub"), options); ``` ```php $capabilities = array( 'platformName' => 'iOS', 'appium:automationName' => 'XCUITest', 'appium:deviceName' => 'iPhone 13', 'appium:platformVersion' => '16.0', 'appium:app' => 'https://testingbot.com/appium/sample.ipa', 'tb:options' => array( 'appiumVersion' => '2.0' ) ); $driver = RemoteWebDriver::create('https://api_key:api_secret@hub.testingbot.com/wd/hub', $capabilities); ``` ```python from appium import webdriver from appium.options.ios import XCUITestOptions capabilities = { "platformName": "iOS", "appium:automationName": "XCUITest", "appium:deviceName": "iPhone 13", "appium:platformVersion": "16.0", "appium:app": "https://testingbot.com/appium/sample.ipa", "tb:options": { "key": "api_key", "secret": "api_secret", "appiumVersion": "2.0" } } options = XCUITestOptions().load_capabilities(capabilities) driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options) ``` ```javascript const { remote } = require('webdriverio'); const capabilities = { "platformName": "iOS", "appium:automationName": "XCUITest", "appium:deviceName": "iPhone 13", "appium:platformVersion": "16.0", "appium:app": "https://testingbot.com/appium/sample.ipa", "tb:options": { "key": "api_key", "secret": "api_secret", "appiumVersion": "2.0" } }; const driver = await remote({ protocol: "https", hostname: "hub.testingbot.com", port: 443, path: "/wd/hub", capabilities }); ``` ```csharp var appiumOptions = new AppiumOptions(); appiumOptions.PlatformName = "iOS"; appiumOptions.AutomationName = "XCUITest"; appiumOptions.AddAdditionalAppiumOption("appium:deviceName", "iPhone 13"); appiumOptions.AddAdditionalAppiumOption("appium:platformVersion", "16.0"); appiumOptions.AddAdditionalAppiumOption("appium:app", "https://testingbot.com/appium/sample.ipa"); var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["appiumVersion"] = "2.0" }; appiumOptions.AddAdditionalAppiumOption("tb:options", tbOptions); var driver = new IOSDriver(new Uri("https://hub.testingbot.com/wd/hub"), appiumOptions, TimeSpan.FromSeconds(600)); ``` ### Appium Versions We support both Appium 3 and Appium 2. A full selectable list is available in the [capabilities builder](https://testingbot.com/support/app-automate/appium/capabilities). Some of the versions we currently support: - 3.2.2 - 3.0.1 - 2.17.0 - 2.11.3 - 2.10.3 - 2.9.0 - 2.5.1 - 2.4.1 If you specify `"appiumVersion": "latest"`, TestingBot will automatically use the latest Appium version. You can also use `latest-1`, `latest-2`, ... to test on the next most recent versions of Appium. ## Change Device Timezone TestingBot allows you to change the timezone of the mobile device (physical or emu/sim) to a timezone that you specify. Simply pass in a `timeZone` capability specifying a timezone from the list. You can find a [list of timezones on Wikipedia](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones). [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { "platformName" => "Android", "appium:automationName" => "UiAutomator2", "appium:deviceName" => "Galaxy S23", "appium:platformVersion" => "13.0", "appium:app" => "https://testingbot.com/appium/sample.apk", "tb:options" => { "timeZone" => "Etc/UTC" } } driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "https://api_key:api_secret@hub.testingbot.com/wd/hub" } }, true).start_driver ``` ```java UiAutomator2Options options = new UiAutomator2Options(); options.setPlatformName("Android"); options.setAutomationName("UiAutomator2"); options.setDeviceName("Galaxy S23"); options.setPlatformVersion("13.0"); options.setApp("https://testingbot.com/appium/sample.apk"); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("timeZone", "Etc/UTC"); options.setCapability("tb:options", tbOptions); AndroidDriver driver = new AndroidDriver( new URL("https://hub.testingbot.com/wd/hub"), options); ``` ```php $capabilities = array( 'platformName' => 'Android', 'appium:automationName' => 'UiAutomator2', 'appium:deviceName' => 'Galaxy S23', 'appium:platformVersion' => '13.0', 'appium:app' => 'https://testingbot.com/appium/sample.apk', 'tb:options' => array( 'timeZone' => 'Etc/UTC' ) ); $driver = RemoteWebDriver::create('https://api_key:api_secret@hub.testingbot.com/wd/hub', $capabilities); ``` ```python from appium import webdriver from appium.options.android import UiAutomator2Options capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "Galaxy S23", "appium:platformVersion": "13.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", "timeZone": "Etc/UTC" } } options = UiAutomator2Options().load_capabilities(capabilities) driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options) ``` ```javascript const { remote } = require('webdriverio'); const capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "Galaxy S23", "appium:platformVersion": "13.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", "timeZone": "Etc/UTC" } }; const driver = await remote({ protocol: "https", hostname: "hub.testingbot.com", port: 443, path: "/wd/hub", capabilities }); ``` ```csharp var appiumOptions = new AppiumOptions(); appiumOptions.PlatformName = "Android"; appiumOptions.AutomationName = "UiAutomator2"; appiumOptions.AddAdditionalAppiumOption("appium:deviceName", "Galaxy S23"); appiumOptions.AddAdditionalAppiumOption("appium:platformVersion", "13.0"); appiumOptions.AddAdditionalAppiumOption("appium:app", "https://testingbot.com/appium/sample.apk"); var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["timeZone"] = "Etc/UTC" }; appiumOptions.AddAdditionalAppiumOption("tb:options", tbOptions); var driver = new AndroidDriver(new Uri("https://hub.testingbot.com/wd/hub"), appiumOptions, TimeSpan.FromSeconds(600)); ``` Value Type | Default Value | Possible Values: || string | "Etc/UTC" | [List of timezones](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) | ### Localization Options Please see the [Localization Options documentation](https://testingbot.com/support/app-automate/appium/localization-options) to learn how to set the locale and language of the device during your automated tests. ### Geolocation Testing TestingBot provides an option where you can specify from which country you'd like to run the test from. Once you specify this option, the device we provision for your test will be configured to use a proxy in the country you specified. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { "platformName" => "Android", "appium:automationName" => "UiAutomator2", "appium:deviceName" => "Galaxy S23", "appium:platformVersion" => "13.0", "appium:app" => "https://testingbot.com/appium/sample.apk", "tb:options" => { "testingbot.geoCountryCode" => "DE" } } driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "https://api_key:api_secret@hub.testingbot.com/wd/hub" } }, true).start_driver ``` ```java UiAutomator2Options options = new UiAutomator2Options(); options.setPlatformName("Android"); options.setAutomationName("UiAutomator2"); options.setDeviceName("Galaxy S23"); options.setPlatformVersion("13.0"); options.setApp("https://testingbot.com/appium/sample.apk"); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("testingbot.geoCountryCode", "DE"); options.setCapability("tb:options", tbOptions); AndroidDriver driver = new AndroidDriver( new URL("https://hub.testingbot.com/wd/hub"), options); ``` ```php $capabilities = array( 'platformName' => 'Android', 'appium:automationName' => 'UiAutomator2', 'appium:deviceName' => 'Galaxy S23', 'appium:platformVersion' => '13.0', 'appium:app' => 'https://testingbot.com/appium/sample.apk', 'tb:options' => array( 'testingbot.geoCountryCode' => 'DE' ) ); $driver = RemoteWebDriver::create('https://api_key:api_secret@hub.testingbot.com/wd/hub', $capabilities); ``` ```python from appium import webdriver from appium.options.android import UiAutomator2Options capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "Galaxy S23", "appium:platformVersion": "13.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", "testingbot.geoCountryCode": "DE" } } options = UiAutomator2Options().load_capabilities(capabilities) driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options) ``` ```javascript const { remote } = require('webdriverio'); const capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "Galaxy S23", "appium:platformVersion": "13.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", "testingbot.geoCountryCode": "DE" } }; const driver = await remote({ protocol: "https", hostname: "hub.testingbot.com", port: 443, path: "/wd/hub", capabilities }); ``` ```csharp var appiumOptions = new AppiumOptions(); appiumOptions.PlatformName = "Android"; appiumOptions.AutomationName = "UiAutomator2"; appiumOptions.AddAdditionalAppiumOption("appium:deviceName", "Galaxy S23"); appiumOptions.AddAdditionalAppiumOption("appium:platformVersion", "13.0"); appiumOptions.AddAdditionalAppiumOption("appium:app", "https://testingbot.com/appium/sample.apk"); var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["testingbot.geoCountryCode"] = "DE" }; appiumOptions.AddAdditionalAppiumOption("tb:options", tbOptions); var driver = new AndroidDriver(new Uri("https://hub.testingbot.com/wd/hub"), appiumOptions, TimeSpan.FromSeconds(600)); ``` Specify `testingbot.geoCountryCode` with one of the following country codes: - **'\*'** : this will take a random country from the list below - **'AU'** : Australia - **'BE'** : Belgium - **'BR'** : Brazil - **'CA'** : Canada - **'CL'** : Chile - **'FR'** : France - **'DE'** : Germany - **'IN'** : India - **'IT'** : Italy - **'JP'** : Japan - **'NO'** : Norway - **'SG'** : Singapore - **'ZA'** : South Africa - **'SE'** : Sweden - **'CH'** : Switzerland - **'AE'** : United Arab Emirates - **'GB'** : United Kingdom - **'US'** : United States **Important:** this does not work on Android 4.4 ## Upload files to device You can upload files to the remote iOS or Android device, prior to starting a test. More information is available on this page: [upload files to device](https://testingbot.com/support/app-automate/appium/upload-files). ## Throttle Network TestingBot provides a throttle network feature, which will simulate specific network conditions during a test. We offer various presets: - **Edge** : - 250kbps download - 150kbps upload - 0% Loss - 300ms Latency - **3G** : - 400kbps download - 100kbps upload - 0% Loss - 100ms Latency - **4G** : - 18000kbps download - 9000kbps upload - 0% Loss - 100ms Latency - **airplane** : Simulates Airplane Mode (no network connectivity) - **disable** : Revert the throttling to no throttling. We also offer the possibility to specify your own `download`, `upload`, `loss` and `latency` values. You can pass a preset or specific options at the start of your test (through the `capabilities`), or during the test (with `execute_script`). **At the start of a test:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { throttle_network: { uploadSpeed: 10 * 1024, downloadSpeed: 10 * 1024, latency: 0, loss: 0 } } # or use a preset instead: caps = { throttle_network: "3G" } ``` ```java UiAutomator2Options options = new UiAutomator2Options(); options.setPlatformName("Android"); options.setAutomationName("UiAutomator2"); options.setApp("https://testingbot.com/appium/sample.apk"); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); // Use a custom object OR a preset string (e.g. tbOptions.setCapability("throttle_network", "3G");), not both. MutableCapabilities throttleOptions = new MutableCapabilities(); throttleOptions.setCapability("uploadSpeed", 10 * 1024); throttleOptions.setCapability("downloadSpeed", 10 * 1024); throttleOptions.setCapability("latency", 0); throttleOptions.setCapability("loss", 0); tbOptions.setCapability("throttle_network", throttleOptions); options.setCapability("tb:options", tbOptions); AndroidDriver driver = new AndroidDriver( new URL("https://hub.testingbot.com/wd/hub"), options); ``` ```php $capabilities = array( 'platformName' => 'Android', 'appium:automationName' => 'UiAutomator2', 'appium:app' => 'https://testingbot.com/appium/sample.apk', // Use a custom array OR a preset string ('throttle_network' => '3G'), not both. 'tb:options' => array( 'throttle_network' => array( 'uploadSpeed' => 10 * 1024, 'downloadSpeed' => 10 * 1024, 'latency' => 0, 'loss' => 0 ) ) ); $driver = RemoteWebDriver::create('https://api_key:api_secret@hub.testingbot.com/wd/hub', $capabilities); ``` ```python from appium import webdriver from appium.options.android import UiAutomator2Options # Use a custom dict OR a preset string ("throttle_network": "3G"), not both. capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", "throttle_network": { "uploadSpeed": 10 * 1024, "downloadSpeed": 10 * 1024, "latency": 0, "loss": 0 } } } options = UiAutomator2Options().load_capabilities(capabilities) driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options) ``` ```javascript const { remote } = require('webdriverio'); const capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", // Use a custom object OR a preset string ("throttle_network": "3G"), not both. "throttle_network": { "uploadSpeed": 10 * 1024, "downloadSpeed": 10 * 1024, "latency": 0, "loss": 0 } } }; const driver = await remote({ protocol: "https", hostname: "hub.testingbot.com", port: 443, path: "/wd/hub", capabilities }); ``` ```csharp var appiumOptions = new AppiumOptions(); appiumOptions.PlatformName = "Android"; appiumOptions.AddAdditionalAppiumOption("appium:app", "https://testingbot.com/appium/sample.apk"); var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["name"] = TestContext.CurrentContext.Test.Name, ["throttle_network"] = "3G" }; appiumOptions.AddAdditionalAppiumOption("tb:options", tbOptions); var driver = new AndroidDriver(new Uri("https://hub.testingbot.com/wd/hub"), appiumOptions, TimeSpan.FromSeconds(600)); ``` **During a test:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby driver.execute_script("tb:throttle", "3G") # or driver.execute_script("tb:throttle", { "downloadSpeed": 10 * 1024, "uploadSpeed": 10 * 1024, "latency": 0, "loss": 0 }) ``` ```java ((JavascriptExecutor) driver).executeScript("tb:throttle", "3G"); // or Map throttleMap = new HashMap<>(); throttleMap.put("downloadSpeed", 10 * 1024); throttleMap.put("uploadSpeed", 10 * 1024); throttleMap.put("latency", 0); throttleMap.put("loss", 0); ((JavascriptExecutor) driver).executeScript("tb:throttle", throttleMap); ``` ```php $driver->executeScript("tb:throttle", ["3G"]); // or $driver->executeScript("tb:throttle", [ "downloadSpeed" => 10 * 1024, "uploadSpeed" => 10 * 1024, "latency" => 0, "loss" => 0 ]); ``` ```python driver.execute_script("tb:throttle", "3G") # or driver.execute_script("tb:throttle", { "downloadSpeed": 10 * 1024, "uploadSpeed": 10 * 1024, "latency": 0, "loss": 0 }) ``` ```javascript browser.execute('tb:throttle', '3G') // or browser.execute('tb:throttle', { "downloadSpeed": 10 * 1024, "uploadSpeed": 10 * 1024, "latency": 0, "loss": 0 }) ``` ```csharp ((IJavaScriptExecutor)driver).ExecuteScript("tb:throttle", "3G"); // or var throttle = new Dictionary { ["downloadSpeed"] = 10 * 1024, ["uploadSpeed"] = 10 * 1024, ["latency"] = 0, ["loss"] = 0 }; ((IJavaScriptExecutor)driver).ExecuteScript("tb:throttle", throttle); ``` To stop network throttling during your test, pass `disable: true` in the `tb:throttle` options via the `execute_script`. ## mobile: shell You can execute ADB shell commands during your automated test, with Appium's `mobile: shell` functionality. This allows you to fetch or set various settings for the device or emulator, during your automated test. For example, send a custom `input` event, or use `dumpsys`, `pm clear` or others. When running a test on an emulator, all ADB shell commands are allowed. For tests running on physical devices, we maintain a whitelist of allowed commands **Whitelisted ADB commands for physical devices** : - am start - am force-stop - pm clear - input - (ls|cp|mkdir|echo|grep|cut|pwd|dumpsys) - getprop - am compat enable - cmd connectivity airplane-mode (enable|disable) - settings put global (animator\_duration\_scale|transition\_animation\_scale|window\_animation\_scale) \ - setprop debug.firebase.analytics.app \ - setprop (log.tag.FA|log.tag.FA-SVC) (ERROR|WARN|INFO|DEBUG|VERBOSE) - setprop (log.tag.FA|log.tag.FA-SVC|log.tag.GAv4|log.tag.GAv4-SVC) (ERROR|WARN|INFO|DEBUG|VERBOSE) - appops set \ REQUEST\_INSTALL\_PACKAGES allow [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby driver.execute_script('mobile: shell', { command: 'input', args: ['keyevent', 3] }) ``` ```java driver.executeScript("mobile: shell", ImmutableMap.of( "command", "input", "args", ImmutableList.of("keyevent", "3") )); ``` ```php $args = [ "command" => "input", "args" => ["keyevent", "3"] ]; $driver->executeScript("mobile: shell", $args); ``` ```python args = { "command": "input", "args": ["keyevent", "3"] } driver.execute_script("mobile: shell", args) ``` ```javascript await driver.executeScript("mobile: shell", { command: "input", args: ["keyevent", "3"] }); ``` ```csharp var args = new Dictionary { { "command", "input" }, { "args", new List { "keyevent", "3" } } }; ((IJavaScriptExecutor)driver).ExecuteScript("mobile: shell", args); ``` ## Selecting the device Capabilities for picking the device, OS version and form factor your Appium test runs on. ### Device Name When running a mobile automated test, you'll need to specify on which [mobile device](https://testingbot.com/support/app-automate/devices) (or [simulator/emulator](https://testingbot.com/support/web-automate/browsers)) you want to test. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { "platformName" => "Android", "appium:automationName" => "UiAutomator2", "appium:deviceName" => "Pixel 9", "appium:platformVersion" => "14.0", "appium:app" => "https://testingbot.com/appium/sample.apk" } driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "https://api_key:api_secret@hub.testingbot.com/wd/hub" } }, true).start_driver ``` ```java UiAutomator2Options options = new UiAutomator2Options(); options.setPlatformName("Android"); options.setAutomationName("UiAutomator2"); options.setDeviceName("Pixel 9"); options.setPlatformVersion("14.0"); options.setApp("https://testingbot.com/appium/sample.apk"); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); options.setCapability("tb:options", tbOptions); AndroidDriver driver = new AndroidDriver( new URL("https://hub.testingbot.com/wd/hub"), options); ``` ```python from appium import webdriver from appium.options.android import UiAutomator2Options capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "Pixel 9", "appium:platformVersion": "14.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret" } } options = UiAutomator2Options().load_capabilities(capabilities) driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options) ``` ```javascript const { remote } = require('webdriverio'); const capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "Pixel 9", "appium:platformVersion": "14.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret" } }; const driver = await remote({ protocol: "https", hostname: "hub.testingbot.com", port: 443, path: "/wd/hub", capabilities }); ``` ```csharp var appiumOptions = new AppiumOptions(); appiumOptions.PlatformName = "Android"; appiumOptions.AutomationName = "UiAutomator2"; appiumOptions.AddAdditionalAppiumOption("appium:deviceName", "Pixel 9"); appiumOptions.AddAdditionalAppiumOption("appium:platformVersion", "14.0"); appiumOptions.AddAdditionalAppiumOption("appium:app", "https://testingbot.com/appium/sample.apk"); var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret" }; appiumOptions.AddAdditionalAppiumOption("tb:options", tbOptions); var driver = new AndroidDriver(new Uri("https://hub.testingbot.com/wd/hub"), appiumOptions, TimeSpan.FromSeconds(600)); ``` Physical Devices Only We offer special parameters which you can use to allocate a device: Regex Input | Result || `"iPhone.*"` | This will allocate any available iPhone device (phone) | | `".*Galaxy.*"` | This will allocate any of the available Galaxy devices (phone or tablet) | | `"*"` | This will allocate a random available device, either iOS or Android device | | `"iPhone [8-11]"` | This will allocate either an iPhone 8 or 11 | | `"iPhone 6.*"` | This will allocate either an iPhone 6 or 6S | Some Examples: ```javascript // find any iPhone, except 6 or 6s capabilities.setCapability("deviceName", "^(iPhone.*)(?!6|6S)$"); // find any device which name starts with Pixel capabilities.setCapability("deviceName", "Pixel.*"); ``` ### Device Version You can specify the version of the device you want to target. In case of physical devices, you might want to use regex/wildcard patterns to target a broad range of devices. This way your chance of hitting an occupied device during your automated test decreases significantly. Instead of only targetting a specific iOS or Android device, you can target a wider range of devices. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { "platformName" => "Android", "appium:automationName" => "UiAutomator2", "appium:deviceName" => "*", "appium:platformVersion" => "(13|14).*", "appium:app" => "https://testingbot.com/appium/sample.apk" } driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "https://api_key:api_secret@hub.testingbot.com/wd/hub" } }, true).start_driver ``` ```java UiAutomator2Options options = new UiAutomator2Options(); options.setPlatformName("Android"); options.setAutomationName("UiAutomator2"); options.setDeviceName("*"); options.setPlatformVersion("(13|14).*"); options.setApp("https://testingbot.com/appium/sample.apk"); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); options.setCapability("tb:options", tbOptions); AndroidDriver driver = new AndroidDriver( new URL("https://hub.testingbot.com/wd/hub"), options); ``` ```python from appium import webdriver from appium.options.android import UiAutomator2Options capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "*", "appium:platformVersion": "(13|14).*", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret" } } options = UiAutomator2Options().load_capabilities(capabilities) driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options) ``` ```javascript const { remote } = require('webdriverio'); const capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "*", "appium:platformVersion": "(13|14).*", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret" } }; const driver = await remote({ protocol: "https", hostname: "hub.testingbot.com", port: 443, path: "/wd/hub", capabilities }); ``` ```csharp var appiumOptions = new AppiumOptions(); appiumOptions.PlatformName = "Android"; appiumOptions.AutomationName = "UiAutomator2"; appiumOptions.AddAdditionalAppiumOption("appium:deviceName", "*"); appiumOptions.AddAdditionalAppiumOption("appium:platformVersion", "(13|14).*"); appiumOptions.AddAdditionalAppiumOption("appium:app", "https://testingbot.com/appium/sample.apk"); var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret" }; appiumOptions.AddAdditionalAppiumOption("tb:options", tbOptions); var driver = new AndroidDriver(new Uri("https://hub.testingbot.com/wd/hub"), appiumOptions, TimeSpan.FromSeconds(600)); ``` Some Examples: ```javascript // target multiple specific versions of an iPhone device capabilities.setCapability("appium:deviceName", "iPhone.*"); capabilities.setCapability("appium:platformVersion", "18.0|15.0|14.2|14"); // find any Pixel device with Android version 15 or 16 capabilities.setCapability("appium:deviceName", "Pixel.*"); capabilities.setCapability("appium:platformVersion", "(15|16).*"); ``` ### Tablet Only You can specify this capability when you only want to allocate a tablet device. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { "platformName" => "Android", "appium:automationName" => "UiAutomator2", "appium:deviceName" => "*", "appium:platformVersion" => "14.0", "appium:app" => "https://testingbot.com/appium/sample.apk", "tb:options" => { "tabletOnly" => true } } driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "https://api_key:api_secret@hub.testingbot.com/wd/hub" } }, true).start_driver ``` ```java UiAutomator2Options options = new UiAutomator2Options(); options.setPlatformName("Android"); options.setAutomationName("UiAutomator2"); options.setDeviceName("*"); options.setPlatformVersion("14.0"); options.setApp("https://testingbot.com/appium/sample.apk"); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("tabletOnly", true); options.setCapability("tb:options", tbOptions); AndroidDriver driver = new AndroidDriver( new URL("https://hub.testingbot.com/wd/hub"), options); ``` ```python from appium import webdriver from appium.options.android import UiAutomator2Options capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "*", "appium:platformVersion": "14.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", "tabletOnly": True } } options = UiAutomator2Options().load_capabilities(capabilities) driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options) ``` ```javascript const { remote } = require('webdriverio'); const capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "*", "appium:platformVersion": "14.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", "tabletOnly": true } }; const driver = await remote({ protocol: "https", hostname: "hub.testingbot.com", port: 443, path: "/wd/hub", capabilities }); ``` ```csharp var appiumOptions = new AppiumOptions(); appiumOptions.PlatformName = "Android"; appiumOptions.AutomationName = "UiAutomator2"; appiumOptions.AddAdditionalAppiumOption("appium:deviceName", "*"); appiumOptions.AddAdditionalAppiumOption("appium:platformVersion", "14.0"); appiumOptions.AddAdditionalAppiumOption("appium:app", "https://testingbot.com/appium/sample.apk"); var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["tabletOnly"] = true }; appiumOptions.AddAdditionalAppiumOption("tb:options", tbOptions); var driver = new AndroidDriver(new Uri("https://hub.testingbot.com/wd/hub"), appiumOptions, TimeSpan.FromSeconds(600)); ``` ### Phone Only You can specify this capability when you only want to allocate a phone device. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { "platformName" => "Android", "appium:automationName" => "UiAutomator2", "appium:deviceName" => "*", "appium:platformVersion" => "14.0", "appium:app" => "https://testingbot.com/appium/sample.apk", "tb:options" => { "phoneOnly" => true } } driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "https://api_key:api_secret@hub.testingbot.com/wd/hub" } }, true).start_driver ``` ```java UiAutomator2Options options = new UiAutomator2Options(); options.setPlatformName("Android"); options.setAutomationName("UiAutomator2"); options.setDeviceName("*"); options.setPlatformVersion("14.0"); options.setApp("https://testingbot.com/appium/sample.apk"); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("phoneOnly", true); options.setCapability("tb:options", tbOptions); AndroidDriver driver = new AndroidDriver( new URL("https://hub.testingbot.com/wd/hub"), options); ``` ```python from appium import webdriver from appium.options.android import UiAutomator2Options capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "*", "appium:platformVersion": "14.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", "phoneOnly": True } } options = UiAutomator2Options().load_capabilities(capabilities) driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options) ``` ```javascript const { remote } = require('webdriverio'); const capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "*", "appium:platformVersion": "14.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", "phoneOnly": true } }; const driver = await remote({ protocol: "https", hostname: "hub.testingbot.com", port: 443, path: "/wd/hub", capabilities }); ``` ```csharp var appiumOptions = new AppiumOptions(); appiumOptions.PlatformName = "Android"; appiumOptions.AutomationName = "UiAutomator2"; appiumOptions.AddAdditionalAppiumOption("appium:deviceName", "*"); appiumOptions.AddAdditionalAppiumOption("appium:platformVersion", "14.0"); appiumOptions.AddAdditionalAppiumOption("appium:app", "https://testingbot.com/appium/sample.apk"); var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["phoneOnly"] = true }; appiumOptions.AddAdditionalAppiumOption("tb:options", tbOptions); var driver = new AndroidDriver(new Uri("https://hub.testingbot.com/wd/hub"), appiumOptions, TimeSpan.FromSeconds(600)); ``` ## Customizing the test Capabilities for naming, grouping, timing out and attaching metadata to your Appium test sessions. ### Change Test Name Add a name to this test, which will show up in the TestingBot member area and API. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { "platformName" => "Android", "appium:automationName" => "UiAutomator2", "appium:deviceName" => "Galaxy S23", "appium:platformVersion" => "13.0", "appium:app" => "https://testingbot.com/appium/sample.apk", "tb:options" => { "name" => "My Test Name" } } driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "https://api_key:api_secret@hub.testingbot.com/wd/hub" } }, true).start_driver ``` ```java UiAutomator2Options options = new UiAutomator2Options(); options.setPlatformName("Android"); options.setAutomationName("UiAutomator2"); options.setDeviceName("Galaxy S23"); options.setPlatformVersion("13.0"); options.setApp("https://testingbot.com/appium/sample.apk"); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("name", "My Test Name"); options.setCapability("tb:options", tbOptions); AndroidDriver driver = new AndroidDriver( new URL("https://hub.testingbot.com/wd/hub"), options); ``` ```php $capabilities = array( 'platformName' => 'Android', 'appium:automationName' => 'UiAutomator2', 'appium:deviceName' => 'Galaxy S23', 'appium:platformVersion' => '13.0', 'appium:app' => 'https://testingbot.com/appium/sample.apk', 'tb:options' => array( 'name' => 'My Test Name' ) ); $driver = RemoteWebDriver::create('https://api_key:api_secret@hub.testingbot.com/wd/hub', $capabilities); ``` ```python from appium import webdriver from appium.options.android import UiAutomator2Options capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "Galaxy S23", "appium:platformVersion": "13.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", "name": "My Test Name" } } options = UiAutomator2Options().load_capabilities(capabilities) driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options) ``` ```javascript const { remote } = require('webdriverio'); const capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "Galaxy S23", "appium:platformVersion": "13.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", "name": "My Test Name" } }; const driver = await remote({ protocol: "https", hostname: "hub.testingbot.com", port: 443, path: "/wd/hub", capabilities }); ``` ```csharp var appiumOptions = new AppiumOptions(); appiumOptions.PlatformName = "Android"; appiumOptions.AutomationName = "UiAutomator2"; appiumOptions.AddAdditionalAppiumOption("appium:deviceName", "Galaxy S23"); appiumOptions.AddAdditionalAppiumOption("appium:platformVersion", "13.0"); appiumOptions.AddAdditionalAppiumOption("appium:app", "https://testingbot.com/appium/sample.apk"); var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["name"] = "My Test Name" }; appiumOptions.AddAdditionalAppiumOption("tb:options", tbOptions); var driver = new AndroidDriver(new Uri("https://hub.testingbot.com/wd/hub"), appiumOptions, TimeSpan.FromSeconds(600)); ``` Value Type | Default Value || string | unnamed test | ### Group Tests A key you can use to group certain tests in the same build (for example in Jenkins). The builds will appear in our member area. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { "platformName" => "Android", "appium:automationName" => "UiAutomator2", "appium:deviceName" => "Galaxy S23", "appium:platformVersion" => "13.0", "appium:app" => "https://testingbot.com/appium/sample.apk", "tb:options" => { "build" => "My First Build" } } driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "https://api_key:api_secret@hub.testingbot.com/wd/hub" } }, true).start_driver ``` ```java UiAutomator2Options options = new UiAutomator2Options(); options.setPlatformName("Android"); options.setAutomationName("UiAutomator2"); options.setDeviceName("Galaxy S23"); options.setPlatformVersion("13.0"); options.setApp("https://testingbot.com/appium/sample.apk"); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("build", "My First Build"); options.setCapability("tb:options", tbOptions); AndroidDriver driver = new AndroidDriver( new URL("https://hub.testingbot.com/wd/hub"), options); ``` ```php $capabilities = array( 'platformName' => 'Android', 'appium:automationName' => 'UiAutomator2', 'appium:deviceName' => 'Galaxy S23', 'appium:platformVersion' => '13.0', 'appium:app' => 'https://testingbot.com/appium/sample.apk', 'tb:options' => array( 'build' => 'My First Build' ) ); $driver = RemoteWebDriver::create('https://api_key:api_secret@hub.testingbot.com/wd/hub', $capabilities); ``` ```python from appium import webdriver from appium.options.android import UiAutomator2Options capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "Galaxy S23", "appium:platformVersion": "13.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", "build": "My First Build" } } options = UiAutomator2Options().load_capabilities(capabilities) driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options) ``` ```javascript const { remote } = require('webdriverio'); const capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "Galaxy S23", "appium:platformVersion": "13.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", "build": "My First Build" } }; const driver = await remote({ protocol: "https", hostname: "hub.testingbot.com", port: 443, path: "/wd/hub", capabilities }); ``` ```csharp var appiumOptions = new AppiumOptions(); appiumOptions.PlatformName = "Android"; appiumOptions.AutomationName = "UiAutomator2"; appiumOptions.AddAdditionalAppiumOption("appium:deviceName", "Galaxy S23"); appiumOptions.AddAdditionalAppiumOption("appium:platformVersion", "13.0"); appiumOptions.AddAdditionalAppiumOption("appium:app", "https://testingbot.com/appium/sample.apk"); var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["build"] = "My First Build" }; appiumOptions.AddAdditionalAppiumOption("tb:options", tbOptions); var driver = new AndroidDriver(new Uri("https://hub.testingbot.com/wd/hub"), appiumOptions, TimeSpan.FromSeconds(600)); ``` Value Type || string | ### Idle Timeout The maximum amount of time a device will wait before proceeding to the next step in your test. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { "platformName" => "Android", "appium:automationName" => "UiAutomator2", "appium:deviceName" => "Galaxy S23", "appium:platformVersion" => "13.0", "appium:app" => "https://testingbot.com/appium/sample.apk", "tb:options" => { "idletimeout" => 130 } } driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "https://api_key:api_secret@hub.testingbot.com/wd/hub" } }, true).start_driver ``` ```java UiAutomator2Options options = new UiAutomator2Options(); options.setPlatformName("Android"); options.setAutomationName("UiAutomator2"); options.setDeviceName("Galaxy S23"); options.setPlatformVersion("13.0"); options.setApp("https://testingbot.com/appium/sample.apk"); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("idletimeout", 130); options.setCapability("tb:options", tbOptions); AndroidDriver driver = new AndroidDriver( new URL("https://hub.testingbot.com/wd/hub"), options); ``` ```php $capabilities = array( 'platformName' => 'Android', 'appium:automationName' => 'UiAutomator2', 'appium:deviceName' => 'Galaxy S23', 'appium:platformVersion' => '13.0', 'appium:app' => 'https://testingbot.com/appium/sample.apk', 'tb:options' => array( 'idletimeout' => 130 ) ); $driver = RemoteWebDriver::create('https://api_key:api_secret@hub.testingbot.com/wd/hub', $capabilities); ``` ```python from appium import webdriver from appium.options.android import UiAutomator2Options capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "Galaxy S23", "appium:platformVersion": "13.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", "idletimeout": 130 } } options = UiAutomator2Options().load_capabilities(capabilities) driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options) ``` ```javascript const { remote } = require('webdriverio'); const capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "Galaxy S23", "appium:platformVersion": "13.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", "idletimeout": 130 } }; const driver = await remote({ protocol: "https", hostname: "hub.testingbot.com", port: 443, path: "/wd/hub", capabilities }); ``` ```csharp var appiumOptions = new AppiumOptions(); appiumOptions.PlatformName = "Android"; appiumOptions.AutomationName = "UiAutomator2"; appiumOptions.AddAdditionalAppiumOption("appium:deviceName", "Galaxy S23"); appiumOptions.AddAdditionalAppiumOption("appium:platformVersion", "13.0"); appiumOptions.AddAdditionalAppiumOption("appium:app", "https://testingbot.com/appium/sample.apk"); var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["idletimeout"] = 130 }; appiumOptions.AddAdditionalAppiumOption("tb:options", tbOptions); var driver = new AndroidDriver(new Uri("https://hub.testingbot.com/wd/hub"), appiumOptions, TimeSpan.FromSeconds(600)); ``` Value Type | Default Value || int (specify number of seconds) | 130 seconds | ### Maximum Test Duration The maximum duration for a single test. This is a safeguard to prevent bad tests from using up your credits. We generally recommend to keep tests short (less than 10 minutes). It's better to split up large tests in smaller individual tests. This keeps your tests fast and allows for more parallelization of your tests. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { "platformName" => "Android", "appium:automationName" => "UiAutomator2", "appium:deviceName" => "Galaxy S23", "appium:platformVersion" => "13.0", "appium:app" => "https://testingbot.com/appium/sample.apk", "tb:options" => { "maxduration" => 1800 } } driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "https://api_key:api_secret@hub.testingbot.com/wd/hub" } }, true).start_driver ``` ```java UiAutomator2Options options = new UiAutomator2Options(); options.setPlatformName("Android"); options.setAutomationName("UiAutomator2"); options.setDeviceName("Galaxy S23"); options.setPlatformVersion("13.0"); options.setApp("https://testingbot.com/appium/sample.apk"); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("maxduration", 1800); options.setCapability("tb:options", tbOptions); AndroidDriver driver = new AndroidDriver( new URL("https://hub.testingbot.com/wd/hub"), options); ``` ```php $capabilities = array( 'platformName' => 'Android', 'appium:automationName' => 'UiAutomator2', 'appium:deviceName' => 'Galaxy S23', 'appium:platformVersion' => '13.0', 'appium:app' => 'https://testingbot.com/appium/sample.apk', 'tb:options' => array( 'maxduration' => 1800 ) ); $driver = RemoteWebDriver::create('https://api_key:api_secret@hub.testingbot.com/wd/hub', $capabilities); ``` ```python from appium import webdriver from appium.options.android import UiAutomator2Options capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "Galaxy S23", "appium:platformVersion": "13.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", "maxduration": 1800 } } options = UiAutomator2Options().load_capabilities(capabilities) driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options) ``` ```javascript const { remote } = require('webdriverio'); const capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "Galaxy S23", "appium:platformVersion": "13.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", "maxduration": 1800 } }; const driver = await remote({ protocol: "https", hostname: "hub.testingbot.com", port: 443, path: "/wd/hub", capabilities }); ``` ```csharp var appiumOptions = new AppiumOptions(); appiumOptions.PlatformName = "Android"; appiumOptions.AutomationName = "UiAutomator2"; appiumOptions.AddAdditionalAppiumOption("appium:deviceName", "Galaxy S23"); appiumOptions.AddAdditionalAppiumOption("appium:platformVersion", "13.0"); appiumOptions.AddAdditionalAppiumOption("appium:app", "https://testingbot.com/appium/sample.apk"); var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["maxduration"] = 1800 }; appiumOptions.AddAdditionalAppiumOption("tb:options", tbOptions); var driver = new AndroidDriver(new Uri("https://hub.testingbot.com/wd/hub"), appiumOptions, TimeSpan.FromSeconds(600)); ``` Value Type | Default Value || int (specify number of seconds) | 1800 seconds (30 minutes) | ### Custom Metadata Send along custom data, for example your release, server, commit hash, ... This will show up on the test detail page in the TestingBot member area. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { "platformName" => "Android", "appium:automationName" => "UiAutomator2", "appium:deviceName" => "Galaxy S23", "appium:platformVersion" => "13.0", "appium:app" => "https://testingbot.com/appium/sample.apk", "tb:options" => { "extra" => "Extra Information" } } driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "https://api_key:api_secret@hub.testingbot.com/wd/hub" } }, true).start_driver ``` ```java UiAutomator2Options options = new UiAutomator2Options(); options.setPlatformName("Android"); options.setAutomationName("UiAutomator2"); options.setDeviceName("Galaxy S23"); options.setPlatformVersion("13.0"); options.setApp("https://testingbot.com/appium/sample.apk"); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("extra", "Extra Information"); options.setCapability("tb:options", tbOptions); AndroidDriver driver = new AndroidDriver( new URL("https://hub.testingbot.com/wd/hub"), options); ``` ```php $capabilities = array( 'platformName' => 'Android', 'appium:automationName' => 'UiAutomator2', 'appium:deviceName' => 'Galaxy S23', 'appium:platformVersion' => '13.0', 'appium:app' => 'https://testingbot.com/appium/sample.apk', 'tb:options' => array( 'extra' => 'Extra Information' ) ); $driver = RemoteWebDriver::create('https://api_key:api_secret@hub.testingbot.com/wd/hub', $capabilities); ``` ```python from appium import webdriver from appium.options.android import UiAutomator2Options capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "Galaxy S23", "appium:platformVersion": "13.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", "extra": "Extra Information" } } options = UiAutomator2Options().load_capabilities(capabilities) driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options) ``` ```javascript const { remote } = require('webdriverio'); const capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "Galaxy S23", "appium:platformVersion": "13.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", "extra": "Extra Information" } }; const driver = await remote({ protocol: "https", hostname: "hub.testingbot.com", port: 443, path: "/wd/hub", capabilities }); ``` ```csharp var appiumOptions = new AppiumOptions(); appiumOptions.PlatformName = "Android"; appiumOptions.AutomationName = "UiAutomator2"; appiumOptions.AddAdditionalAppiumOption("appium:deviceName", "Galaxy S23"); appiumOptions.AddAdditionalAppiumOption("appium:platformVersion", "13.0"); appiumOptions.AddAdditionalAppiumOption("appium:app", "https://testingbot.com/appium/sample.apk"); var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["extra"] = "Extra Information" }; appiumOptions.AddAdditionalAppiumOption("tb:options", tbOptions); var driver = new AndroidDriver(new Uri("https://hub.testingbot.com/wd/hub"), appiumOptions, TimeSpan.FromSeconds(600)); ``` Value Type || string | ## Privacy and session recording Capabilities for controlling screenshots, video recording, log capture and dashboard privacy. ### Taking screenshots during your tests By default, TestingBot does not capture screenshots at every step of your mobile test. If you wish to take a screenshot for every step, please add this capability (set to `true`) to your request. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { "platformName" => "Android", "appium:automationName" => "UiAutomator2", "appium:deviceName" => "Galaxy S23", "appium:platformVersion" => "13.0", "appium:app" => "https://testingbot.com/appium/sample.apk", "tb:options" => { "screenshot" => true } } driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "https://api_key:api_secret@hub.testingbot.com/wd/hub" } }, true).start_driver ``` ```java UiAutomator2Options options = new UiAutomator2Options(); options.setPlatformName("Android"); options.setAutomationName("UiAutomator2"); options.setDeviceName("Galaxy S23"); options.setPlatformVersion("13.0"); options.setApp("https://testingbot.com/appium/sample.apk"); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("screenshot", true); options.setCapability("tb:options", tbOptions); AndroidDriver driver = new AndroidDriver( new URL("https://hub.testingbot.com/wd/hub"), options); ``` ```php $capabilities = array( 'platformName' => 'Android', 'appium:automationName' => 'UiAutomator2', 'appium:deviceName' => 'Galaxy S23', 'appium:platformVersion' => '13.0', 'appium:app' => 'https://testingbot.com/appium/sample.apk', 'tb:options' => array( 'screenshot' => true ) ); $driver = RemoteWebDriver::create('https://api_key:api_secret@hub.testingbot.com/wd/hub', $capabilities); ``` ```python from appium import webdriver from appium.options.android import UiAutomator2Options capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "Galaxy S23", "appium:platformVersion": "13.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", "screenshot": True } } options = UiAutomator2Options().load_capabilities(capabilities) driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options) ``` ```javascript const { remote } = require('webdriverio'); const capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "Galaxy S23", "appium:platformVersion": "13.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", "screenshot": true } }; const driver = await remote({ protocol: "https", hostname: "hub.testingbot.com", port: 443, path: "/wd/hub", capabilities }); ``` ```csharp var appiumOptions = new AppiumOptions(); appiumOptions.PlatformName = "Android"; appiumOptions.AutomationName = "UiAutomator2"; appiumOptions.AddAdditionalAppiumOption("appium:deviceName", "Galaxy S23"); appiumOptions.AddAdditionalAppiumOption("appium:platformVersion", "13.0"); appiumOptions.AddAdditionalAppiumOption("appium:app", "https://testingbot.com/appium/sample.apk"); var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["screenshot"] = true }; appiumOptions.AddAdditionalAppiumOption("tb:options", tbOptions); var driver = new AndroidDriver(new Uri("https://hub.testingbot.com/wd/hub"), appiumOptions, TimeSpan.FromSeconds(600)); ``` Value Type | Default Value || boolean | false | ### Make a video of your tests By default we record a video of your test, which is accessible in the member area. If you do not wish to have this, you can disable it with this option. Video should not slow down your test considerably. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { "platformName" => "Android", "appium:automationName" => "UiAutomator2", "appium:deviceName" => "Galaxy S23", "appium:platformVersion" => "13.0", "appium:app" => "https://testingbot.com/appium/sample.apk", "tb:options" => { "screenrecorder" => true } } driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "https://api_key:api_secret@hub.testingbot.com/wd/hub" } }, true).start_driver ``` ```java UiAutomator2Options options = new UiAutomator2Options(); options.setPlatformName("Android"); options.setAutomationName("UiAutomator2"); options.setDeviceName("Galaxy S23"); options.setPlatformVersion("13.0"); options.setApp("https://testingbot.com/appium/sample.apk"); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("screenrecorder", true); options.setCapability("tb:options", tbOptions); AndroidDriver driver = new AndroidDriver( new URL("https://hub.testingbot.com/wd/hub"), options); ``` ```php $capabilities = array( 'platformName' => 'Android', 'appium:automationName' => 'UiAutomator2', 'appium:deviceName' => 'Galaxy S23', 'appium:platformVersion' => '13.0', 'appium:app' => 'https://testingbot.com/appium/sample.apk', 'tb:options' => array( 'screenrecorder' => true ) ); $driver = RemoteWebDriver::create('https://api_key:api_secret@hub.testingbot.com/wd/hub', $capabilities); ``` ```python from appium import webdriver from appium.options.android import UiAutomator2Options capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "Galaxy S23", "appium:platformVersion": "13.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", "screenrecorder": True } } options = UiAutomator2Options().load_capabilities(capabilities) driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options) ``` ```javascript const { remote } = require('webdriverio'); const capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "Galaxy S23", "appium:platformVersion": "13.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", "screenrecorder": true } }; const driver = await remote({ protocol: "https", hostname: "hub.testingbot.com", port: 443, path: "/wd/hub", capabilities }); ``` ```csharp var appiumOptions = new AppiumOptions(); appiumOptions.PlatformName = "Android"; appiumOptions.AutomationName = "UiAutomator2"; appiumOptions.AddAdditionalAppiumOption("appium:deviceName", "Galaxy S23"); appiumOptions.AddAdditionalAppiumOption("appium:platformVersion", "13.0"); appiumOptions.AddAdditionalAppiumOption("appium:app", "https://testingbot.com/appium/sample.apk"); var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["screenrecorder"] = true }; appiumOptions.AddAdditionalAppiumOption("tb:options", tbOptions); var driver = new AndroidDriver(new Uri("https://hub.testingbot.com/wd/hub"), appiumOptions, TimeSpan.FromSeconds(600)); ``` Value Type | Default Value || boolean | true | ### Test Privacy Make the test results for this test public so that everyone can access the results. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { "platformName" => "Android", "appium:automationName" => "UiAutomator2", "appium:deviceName" => "Galaxy S23", "appium:platformVersion" => "13.0", "appium:app" => "https://testingbot.com/appium/sample.apk", "tb:options" => { "public" => false } } driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "https://api_key:api_secret@hub.testingbot.com/wd/hub" } }, true).start_driver ``` ```java UiAutomator2Options options = new UiAutomator2Options(); options.setPlatformName("Android"); options.setAutomationName("UiAutomator2"); options.setDeviceName("Galaxy S23"); options.setPlatformVersion("13.0"); options.setApp("https://testingbot.com/appium/sample.apk"); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("public", false); options.setCapability("tb:options", tbOptions); AndroidDriver driver = new AndroidDriver( new URL("https://hub.testingbot.com/wd/hub"), options); ``` ```php $capabilities = array( 'platformName' => 'Android', 'appium:automationName' => 'UiAutomator2', 'appium:deviceName' => 'Galaxy S23', 'appium:platformVersion' => '13.0', 'appium:app' => 'https://testingbot.com/appium/sample.apk', 'tb:options' => array( 'public' => false ) ); $driver = RemoteWebDriver::create('https://api_key:api_secret@hub.testingbot.com/wd/hub', $capabilities); ``` ```python from appium import webdriver from appium.options.android import UiAutomator2Options capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "Galaxy S23", "appium:platformVersion": "13.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", "public": False } } options = UiAutomator2Options().load_capabilities(capabilities) driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options) ``` ```javascript const { remote } = require('webdriverio'); const capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "Galaxy S23", "appium:platformVersion": "13.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", "public": false } }; const driver = await remote({ protocol: "https", hostname: "hub.testingbot.com", port: 443, path: "/wd/hub", capabilities }); ``` ```csharp var appiumOptions = new AppiumOptions(); appiumOptions.PlatformName = "Android"; appiumOptions.AutomationName = "UiAutomator2"; appiumOptions.AddAdditionalAppiumOption("appium:deviceName", "Galaxy S23"); appiumOptions.AddAdditionalAppiumOption("appium:platformVersion", "13.0"); appiumOptions.AddAdditionalAppiumOption("appium:app", "https://testingbot.com/appium/sample.apk"); var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["public"] = false }; appiumOptions.AddAdditionalAppiumOption("tb:options", tbOptions); var driver = new AndroidDriver(new Uri("https://hub.testingbot.com/wd/hub"), appiumOptions, TimeSpan.FromSeconds(600)); ``` Value Type | Default Value || boolean | false | ### Customize Logging By default, TestingBot records logs of all test actions and its drivers. Set this option to `false` if you don't want TestingBot to record anything (for example, if you have sensitive data). You will not see any test logs in our member dashboard. Set to `strip-parameters` to prevent the POST/GET parameters from being logged on the TestingBot test detail page (does not affect other logs like Appium logs, Chromedriver logs, ...). [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { "platformName" => "Android", "appium:automationName" => "UiAutomator2", "appium:deviceName" => "Galaxy S23", "appium:platformVersion" => "13.0", "appium:app" => "https://testingbot.com/appium/sample.apk", "tb:options" => { "recordLogs" => true } } driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "https://api_key:api_secret@hub.testingbot.com/wd/hub" } }, true).start_driver ``` ```java UiAutomator2Options options = new UiAutomator2Options(); options.setPlatformName("Android"); options.setAutomationName("UiAutomator2"); options.setDeviceName("Galaxy S23"); options.setPlatformVersion("13.0"); options.setApp("https://testingbot.com/appium/sample.apk"); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("recordLogs", true); options.setCapability("tb:options", tbOptions); AndroidDriver driver = new AndroidDriver( new URL("https://hub.testingbot.com/wd/hub"), options); ``` ```php $capabilities = array( 'platformName' => 'Android', 'appium:automationName' => 'UiAutomator2', 'appium:deviceName' => 'Galaxy S23', 'appium:platformVersion' => '13.0', 'appium:app' => 'https://testingbot.com/appium/sample.apk', 'tb:options' => array( 'recordLogs' => true ) ); $driver = RemoteWebDriver::create('https://api_key:api_secret@hub.testingbot.com/wd/hub', $capabilities); ``` ```python from appium import webdriver from appium.options.android import UiAutomator2Options capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "Galaxy S23", "appium:platformVersion": "13.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", "recordLogs": True } } options = UiAutomator2Options().load_capabilities(capabilities) driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options) ``` ```javascript const { remote } = require('webdriverio'); const capabilities = { "platformName": "Android", "appium:automationName": "UiAutomator2", "appium:deviceName": "Galaxy S23", "appium:platformVersion": "13.0", "appium:app": "https://testingbot.com/appium/sample.apk", "tb:options": { "key": "api_key", "secret": "api_secret", "recordLogs": true } }; const driver = await remote({ protocol: "https", hostname: "hub.testingbot.com", port: 443, path: "/wd/hub", capabilities }); ``` ```csharp var appiumOptions = new AppiumOptions(); appiumOptions.PlatformName = "Android"; appiumOptions.AutomationName = "UiAutomator2"; appiumOptions.AddAdditionalAppiumOption("appium:deviceName", "Galaxy S23"); appiumOptions.AddAdditionalAppiumOption("appium:platformVersion", "13.0"); appiumOptions.AddAdditionalAppiumOption("appium:app", "https://testingbot.com/appium/sample.apk"); var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["recordLogs"] = true }; appiumOptions.AddAdditionalAppiumOption("tb:options", tbOptions); var driver = new AndroidDriver(new Uri("https://hub.testingbot.com/wd/hub"), appiumOptions, TimeSpan.FromSeconds(600)); ``` Value Type | Default Value | Possible Values: || string | "true" | `true`, `false`, or `strip-parameters` | ## Frequently asked questions How do I select a specific device for my Appium test? Set the `appium:deviceName` capability (for example `"Pixel 9"`) plus `platformName` and `appium:platformVersion`. See the [Device Name section](https://testingbot.com#devicename) for code snippets in Java, Python, Node.js, Ruby and C#. How do I change the test name shown in the TestingBot dashboard? Set `name` inside `tb:options`. See the [Change Test Name section](https://testingbot.com#name) for examples per language. How do I group multiple Appium tests under one build? Set `build` inside `tb:options` with the same identifier across all tests. See the [Group Tests section](https://testingbot.com#build) for examples. What is the default Appium idle timeout on TestingBot? 130 seconds. The session ends if no Appium command is received within that window. Override with the `idleTimeout` capability inside `tb:options`, see the [Idle Timeout section](https://testingbot.com#idletimeout). What is the default maximum Appium test duration? 1800 seconds (30 minutes). After that the session is terminated. Override with `maxDuration` inside `tb:options`, see the [Maximum Test Duration section](https://testingbot.com#maxduration). How do I attach custom metadata to my Appium test? Pass a `custom-data` object inside `tb:options`. The dashboard and the REST API expose it back unchanged. See the [Custom Metadata section](https://testingbot.com#customdata). Can I disable screenshots or screen recording for privacy? Yes. Set `screenshot: false` and/or `screenrecorder: false` inside `tb:options`. For sensitive tests use the `private` capability to redact session metadata. See the [Privacy section](https://testingbot.com#privacy). Can I change the timezone, locale or geolocation of the device? Yes. Pass `timeZone`, `appium:locale`, `appium:language` or `geoCountryCode` inside the capability map. See the [timezone section](https://testingbot.com#timezone) and [geolocation section](https://testingbot.com#geo). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/appium/instrumentation # App Instrumentation with Appium App instrumentation means adding extra hooks to your app while it runs on TestingBot's devices. It lets your automated tests interact with things that normally require human interaction, such as authentication, taking a specific photo or making a call. With app instrumentation, you can perform the following tasks in an automated Appium test script: - [Inject images into the camera](https://testingbot.com#image-injection): feed a test image to the virtual camera instead of using a real one. - [Biometric authentication](https://testingbot.com#biometric): simulate fingerprint or Touch ID prompts in your tests. - [Receive calls and SMS messages](https://testingbot.com#calls-sms): emulate incoming phone calls or text messages to see how your app responds. ## Injecting images into camera Android Emulators Only Image injection allows you to specify a base64 encoded image to the virtual camera of the Android emulator. The test script will fetch the content of the image, convert it to base64 and send it to the TestingBot grid. The image will be used by the remote device to inject into the camera. Make sure to include the `images` Appium plugin, as it is required for this to function correctly. [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) [Ruby](https://testingbot.com#) [C#](https://testingbot.com#) ```java import io.appium.java_client.AppiumDriver; import org.openqa.selenium.remote.DesiredCapabilities; import java.net.URL; import java.util.Base64; import java.nio.file.Files; import java.nio.file.Paths; import java.util.HashMap; public class ImageInjectionExample { public static String imageToBase64(String url) throws Exception { byte[] data = java.net.URI.create(url).toURL().openStream().readAllBytes(); return Base64.getEncoder().encodeToString(data); } public static void main(String[] args) throws Exception { DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("platformName", "Android"); caps.setCapability("browserName", "chrome"); caps.setCapability("browserVersion", "15"); caps.setCapability("appium:deviceName", "Pixel 9"); caps.setCapability("tb:appiumPlugins", new String[]{"images"}); caps.setCapability("tb:appiumVersion", "2.11.3"); caps.setCapability("appium:injectedImageProperties", new HashMap<>()); AppiumDriver driver = new AppiumDriver(new URL("https://key:secret@hub.testingbot.com/wd/hub"), caps); String base64Image = imageToBase64("https://upload.wikimedia.org/wikipedia/commons/5/5e/QR_Code_example.png"); driver.executeScript("mobile: injectEmulatorCameraImage", new HashMap() {{ put("payload", base64Image); }}); Thread.sleep(30000); base64Image = imageToBase64("https://testingbot.com/assets/about.png"); driver.executeScript("mobile: injectEmulatorCameraImage", new HashMap() {{ put("payload", base64Image); }}); Thread.sleep(30000); driver.quit(); } } ``` ```javascript const { remote } = require('webdriverio'); async function imageToBase64(url) { const res = await fetch(url); const buffer = Buffer.from(await res.arrayBuffer()); return buffer.toString('base64'); } (async () => { const caps = { platformName: 'Android', 'appium:deviceName': 'Pixel 9', browserName: 'chrome', browserVersion: '15', 'tb:options': { 'appiumVersion': '2.11.3', 'appiumPlugins': ['images'] }, 'appium:injectedImageProperties': {} }; const driver = await remote({ protocol: 'https', hostname: 'hub.testingbot.com', port: 443, path: '/wd/hub', user: 'key', key: 'secret', capabilities: caps }); let base64Image = await imageToBase64('https://upload.wikimedia.org/wikipedia/commons/5/5e/QR_Code_example.png'); await driver.executeScript('mobile: injectEmulatorCameraImage', [{ payload: base64Image }]); console.log("Injected first image"); await driver.pause(30000); base64Image = await imageToBase64('https://testingbot.com/assets/about.png'); await driver.executeScript('mobile: injectEmulatorCameraImage', [{ payload: base64Image }]); console.log("Injected second image"); await driver.pause(30000); await driver.deleteSession(); })(); ``` ```python from appium import webdriver from appium.options.android import UiAutomator2Options import base64 import requests import time def image_to_base64(url): response = requests.get(url) return base64.b64encode(response.content).decode('utf-8') options = UiAutomator2Options().load_capabilities({ "platformName": "Android", "browserName": "chrome", "browserVersion": "15", "appium:deviceName": "Pixel 9", "appium:injectedImageProperties": {}, "tb:options": { "appiumVersion": "2.11.3", "appiumPlugins": ["images"] } }) driver = webdriver.Remote("https://key:secret@hub.testingbot.com/wd/hub", options=options) base64_image = image_to_base64("https://upload.wikimedia.org/wikipedia/commons/5/5e/QR_Code_example.png") driver.execute_script("mobile: injectEmulatorCameraImage", { "payload": base64_image }) time.sleep(30) print("Changing image") base64_image = image_to_base64("https://testingbot.com/assets/about.png") driver.execute_script("mobile: injectEmulatorCameraImage", { "payload": base64_image }) time.sleep(30) driver.quit() ``` ```ruby #!/usr/bin/env ruby require 'rubygems' require 'appium_lib' require 'base64' require 'open-uri' def image_to_base64(image_url) image_data = URI.parse(image_url).open.read Base64.strict_encode64(image_data) end caps = { "browserName" => "chrome", "browserVersion" => "15", "appium:deviceName" => "Pixel 9", "platformName" => "Android", "tb:appiumPlugins" => ["images"], "tb:appiumVersion" => "2.11.3", "appium:injectedImageProperties" => {} } appium_driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "https://key:secret@hub.testingbot.com/wd/hub" } }, true) driver = appium_driver.start_driver base64_image = image_to_base64("https://upload.wikimedia.org/wikipedia/commons/5/5e/QR_Code_example.png") driver.execute_script('mobile: injectEmulatorCameraImage', { payload: base64_image }) sleep 30 puts "Changing image" base64_image = image_to_base64("https://testingbot.com/assets/about.png") driver.execute_script('mobile: injectEmulatorCameraImage', { payload: base64_image }) sleep 30 driver.quit ``` ```csharp using OpenQA.Selenium; using OpenQA.Selenium.Appium; using OpenQA.Selenium.Remote; using System; using System.Collections.Generic; using System.Net.Http; using System.Threading.Tasks; class ImageInjectionExample { static async Task ImageToBase64(string url) { using (HttpClient client = new HttpClient()) { var data = await client.GetByteArrayAsync(url); return Convert.ToBase64String(data); } } static async Task Main(string[] args) { var options = new AppiumOptions(); options.PlatformName = "Android"; options.AddAdditionalAppiumOption("browserName", "chrome"); options.AddAdditionalAppiumOption("browserVersion", "15"); options.AddAdditionalAppiumOption("appium:deviceName", "Pixel 9"); options.AddAdditionalAppiumOption("tb:appiumPlugins", new string[] { "images" }); options.AddAdditionalAppiumOption("tb:appiumVersion", "2.11.3"); options.AddAdditionalAppiumOption("appium:injectedImageProperties", new { }); var driver = new RemoteWebDriver( new Uri("https://key:secret@hub.testingbot.com/wd/hub"), options ); var base64Image = await ImageToBase64("https://upload.wikimedia.org/wikipedia/commons/5/5e/QR_Code_example.png"); driver.ExecuteScript("mobile: injectEmulatorCameraImage", new Dictionary { { "payload", base64Image } }); Console.WriteLine("Injected first image"); System.Threading.Thread.Sleep(30000); base64Image = await ImageToBase64("https://testingbot.com/assets/about.png"); driver.ExecuteScript("mobile: injectEmulatorCameraImage", new Dictionary { { "payload", base64Image } }); Console.WriteLine("Injected second image"); System.Threading.Thread.Sleep(30000); driver.Quit(); } } ``` ## Biometric authentication (Finger Print, Touch Id) ### Android fingerprint authentication Emulate fingerprints on Android Emulator, with the [mobile:fingerprint](https://github.com/appium/appium-uiautomator2-driver?tab=readme-ov-file#mobile-fingerprint) command. Only works on API 23+ with [Appium version 2 or higher](https://testingbot.com/support/app-automate/appium/appium-versions). [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) [Ruby](https://testingbot.com#) [C#](https://testingbot.com#) ```java import io.appium.java_client.AppiumDriver; import org.openqa.selenium.remote.DesiredCapabilities; import java.net.URL; import java.util.HashMap; public class FingerprintExample { public static void main(String[] args) throws Exception { DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("platformName", "Android"); caps.setCapability("appium:deviceName", "Pixel 9"); caps.setCapability("browserName", "chrome"); caps.setCapability("browserVersion", "15"); caps.setCapability("tb:appiumVersion", "2.11.3"); AppiumDriver driver = new AppiumDriver(new URL("https://key:secret@hub.testingbot.com/wd/hub"), caps); HashMap args1 = new HashMap<>(); args1.put("fingerprintId", 1); driver.executeScript("mobile: fingerprint", args1); Thread.sleep(5000); driver.quit(); } } ``` ```javascript const { remote } = require('webdriverio'); (async () => { const caps = { platformName: 'Android', 'appium:deviceName': 'Pixel 9', browserName: 'chrome', browserVersion: '15', 'tb:appiumVersion': '2.11.3' }; const driver = await remote({ protocol: 'https', hostname: 'hub.testingbot.com', port: 443, path: '/wd/hub', user: 'key', key: 'secret', capabilities: caps }); await driver.executeScript('mobile: fingerprint', [{ fingerprintId: 1 }]); await driver.pause(5000); await driver.deleteSession(); })(); ``` ```python from appium import webdriver from appium.options.android import UiAutomator2Options import time options = UiAutomator2Options().load_capabilities({ "platformName": "Android", "appium:deviceName": "Pixel 9", "browserName": "chrome", "browserVersion": "15", "tb:options": {"appiumVersion": "2.11.3"} }) driver = webdriver.Remote("https://key:secret@hub.testingbot.com/wd/hub", options=options) # Simulate fingerprint auth with ID 1 driver.execute_script("mobile: fingerprint", { "fingerprintId": 1 }) time.sleep(5) driver.quit() ``` ```ruby require 'appium_lib' caps = { "platformName" => "Android", "appium:deviceName" => "Pixel 9", "browserName" => "chrome", "browserVersion" => "15", "tb:appiumVersion" => "2.11.3" } driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "https://key:secret@hub.testingbot.com/wd/hub" } }, true).start_driver driver.execute_script("mobile: fingerprint", { fingerprintId: 1 }) sleep 5 driver.quit ``` ```csharp using OpenQA.Selenium.Remote; using System; using System.Collections.Generic; using OpenQA.Selenium.Appium; class FingerprintExample { static void Main(string[] args) { var options = new AppiumOptions(); options.PlatformName = "Android"; options.AddAdditionalAppiumOption("appium:deviceName", "Pixel 9"); options.AddAdditionalAppiumOption("browserName", "chrome"); options.AddAdditionalAppiumOption("browserVersion", "15"); options.AddAdditionalAppiumOption("tb:appiumVersion", "2.11.3"); var driver = new RemoteWebDriver( new Uri("https://key:secret@hub.testingbot.com/wd/hub"), options); driver.ExecuteScript("mobile: fingerprint", new Dictionary { { "fingerprintId", 1 } }); System.Threading.Thread.Sleep(5000); driver.Quit(); } } ``` ### iOS TouchID/FaceID authentication You can simulate a biometric match/non-match event with TouchID and FaceID, with the [mobile: sendBiometricMatch](https://appium.github.io/appium-xcuitest-driver/latest/reference/execute-methods/#mobile-sendbiometricmatch) command. [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) [Ruby](https://testingbot.com#) [C#](https://testingbot.com#) ```java import io.appium.java_client.AppiumDriver; import org.openqa.selenium.remote.DesiredCapabilities; import java.net.URL; import java.util.HashMap; public class TouchIDExample { public static void main(String[] args) throws Exception { DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("platformName", "iOS"); caps.setCapability("appium:deviceName", "iPhone 15"); caps.setCapability("appium:platformVersion", "18.0"); caps.setCapability("browserName", "safari"); caps.setCapability("tb:appiumVersion", "2.11.3"); AppiumDriver driver = new AppiumDriver(new URL("https://key:secret@hub.testingbot.com/wd/hub"), caps); HashMap args1 = new HashMap<>(); args1.put("type", "touchId"); args1.put("match", true); driver.executeScript("mobile: sendBiometricMatch", args1); Thread.sleep(5000); HashMap args2 = new HashMap<>(); args2.put("type", "touchId"); args2.put("match", false); driver.executeScript("mobile: sendBiometricMatch", args2); Thread.sleep(5000); driver.quit(); } } ``` ```javascript const { remote } = require('webdriverio'); (async () => { const caps = { platformName: 'iOS', 'appium:deviceName': 'iPhone 15', 'appium:platformVersion': '18.0', browserName: 'safari', 'tb:appiumVersion': '2.11.3' }; const driver = await remote({ protocol: 'https', hostname: 'hub.testingbot.com', port: 443, path: '/wd/hub', user: 'key', key: 'secret', capabilities: caps }); // Successful Touch ID await driver.executeScript('mobile: sendBiometricMatch', [{ type: 'touchId', match: true }]); await driver.pause(5000); // Failed Touch ID await driver.executeScript('mobile: sendBiometricMatch', [{ type: 'touchId', match: false }]); await driver.pause(5000); await driver.deleteSession(); })(); ``` ```python from appium import webdriver from appium.options.ios import XCUITestOptions import time options = XCUITestOptions().load_capabilities({ "platformName": "iOS", "appium:deviceName": "iPhone 15", "appium:platformVersion": "18.0", "browserName": "safari", "tb:options": {"appiumVersion": "2.11.3"} }) driver = webdriver.Remote("https://key:secret@hub.testingbot.com/wd/hub", options=options) # Successful Touch ID driver.execute_script("mobile: sendBiometricMatch", { "type": "touchId", "match": True }) time.sleep(5) # Failed Touch ID driver.execute_script("mobile: sendBiometricMatch", { "type": "touchId", "match": False }) time.sleep(5) driver.quit() ``` ```ruby require 'appium_lib' caps = { "platformName" => "iOS", "appium:deviceName" => "iPhone 15", "appium:platformVersion" => "18.0", "browserName" => "safari", "tb:appiumVersion" => "2.11.3" } driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "https://key:secret@hub.testingbot.com/wd/hub" } }, true).start_driver # Success driver.execute_script("mobile: sendBiometricMatch", { type: "touchId", match: true }) sleep 5 # Failure driver.execute_script("mobile: sendBiometricMatch", { type: "touchId", match: false }) sleep 5 driver.quit ``` ```csharp using OpenQA.Selenium.Remote; using System; using System.Collections.Generic; using OpenQA.Selenium.Appium; class TouchIDExample { static void Main(string[] args) { var options = new AppiumOptions(); options.PlatformName = "iOS"; options.AddAdditionalAppiumOption("appium:deviceName", "iPhone 15"); options.AddAdditionalAppiumOption("appium:platformVersion", "18.0"); options.AddAdditionalAppiumOption("browserName", "safari"); options.AddAdditionalAppiumOption("tb:appiumVersion", "2.11.3"); var driver = new RemoteWebDriver( new Uri("https://key:secret@hub.testingbot.com/wd/hub"), options); driver.ExecuteScript("mobile: sendBiometricMatch", new Dictionary { { "type", "touchId" }, { "match", true } }); System.Threading.Thread.Sleep(5000); driver.ExecuteScript("mobile: sendBiometricMatch", new Dictionary { { "type", "touchId" }, { "match", false } }); System.Threading.Thread.Sleep(5000); driver.Quit(); } } ``` ## Receiving calls and text messages (SMS) You can simulate an incoming call or text message with Appium. Please see the examples below. [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) [Ruby](https://testingbot.com#) [C#](https://testingbot.com#) ```java import io.appium.java_client.AppiumDriver; import io.appium.java_client.android.AndroidDriver; import io.appium.java_client.android.connection.GsmCallActions; import org.openqa.selenium.remote.DesiredCapabilities; import java.net.URL; public class AndroidGsmExample { public static void main(String[] args) throws Exception { DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("platformName", "Android"); caps.setCapability("appium:deviceName", "Pixel 9"); caps.setCapability("browserName", "chrome"); caps.setCapability("browserVersion", "15"); caps.setCapability("tb:appiumVersion", "2.11.3"); AndroidDriver driver = new AndroidDriver(new URL("https://key:secret@hub.testingbot.com/wd/hub"), caps); // Send SMS driver.sendSMS("5555555555", "Your message here!"); // Make and handle a phone call driver.makeGsmCall("5555555555", GsmCallActions.CALL); Thread.sleep(2000); driver.makeGsmCall("5555555555", GsmCallActions.ACCEPT); Thread.sleep(2000); driver.makeGsmCall("5555555555", GsmCallActions.CANCEL); driver.quit(); } } ``` ```javascript const { remote } = require('webdriverio'); (async () => { const caps = { platformName: 'Android', 'appium:deviceName': 'Pixel 9', browserName: 'chrome', browserVersion: '15', 'tb:appiumVersion': '2.11.3' }; const driver = await remote({ protocol: 'https', hostname: 'hub.testingbot.com', port: 443, path: '/wd/hub', user: 'key', key: 'secret', capabilities: caps }); // Send SMS await driver.sendSms('5555555555', 'Your message here!'); // Phone call await driver.makeGsmCall('5555555555', 'call'); await driver.pause(2000); await driver.makeGsmCall('5555555555', 'accept'); await driver.pause(2000); await driver.makeGsmCall('5555555555', 'cancel'); await driver.deleteSession(); })(); ``` ```python from appium import webdriver from appium.options.android import UiAutomator2Options import time options = UiAutomator2Options().load_capabilities({ "platformName": "Android", "appium:deviceName": "Pixel 9", "browserName": "chrome", "browserVersion": "15", "tb:options": {"appiumVersion": "2.11.3"} }) driver = webdriver.Remote("https://key:secret@hub.testingbot.com/wd/hub", options=options) # Send SMS driver.send_sms("5555555555", "Your message here!") # Phone call driver.make_gsm_call("5555555555", "call") time.sleep(2) driver.make_gsm_call("5555555555", "accept") time.sleep(2) driver.make_gsm_call("5555555555", "cancel") driver.quit() ``` ```ruby require 'appium_lib' caps = { "platformName" => "Android", "appium:deviceName" => "Pixel 9", "browserName" => "chrome", "browserVersion" => "15", "tb:appiumVersion" => "2.11.3" } driver = Appium::Driver.new({ caps: caps, appium_lib: { server_url: "https://key:secret@hub.testingbot.com/wd/hub" } }, true).start_driver # Send SMS driver.send_sms("5555555555", "Your message here!") # Phone call driver.make_gsm_call("5555555555", "call") sleep 2 driver.make_gsm_call("5555555555", "accept") sleep 2 driver.make_gsm_call("5555555555", "cancel") driver.quit ``` ```csharp using OpenQA.Selenium.Remote; using OpenQA.Selenium.Appium; using OpenQA.Selenium.Appium.Android; using System; class AndroidGsmExample { static void Main(string[] args) { var options = new AppiumOptions(); options.PlatformName = "Android"; options.AddAdditionalAppiumOption("appium:deviceName", "Pixel 9"); options.AddAdditionalAppiumOption("browserName", "chrome"); options.AddAdditionalAppiumOption("browserVersion", "15"); options.AddAdditionalAppiumOption("tb:appiumVersion", "2.11.3"); var driver = new AndroidDriver(new Uri("https://key:secret@hub.testingbot.com/wd/hub"), options); // Send SMS driver.SendSMS("5555555555", "Your message here!"); // Phone call driver.MakeGsmCall("5555555555", "call"); System.Threading.Thread.Sleep(2000); driver.MakeGsmCall("5555555555", "accept"); System.Threading.Thread.Sleep(2000); driver.MakeGsmCall("5555555555", "cancel"); driver.Quit(); } } ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/xcuitest # XCUITest Intro TestingBot provides a grid of physical iOS devices in the cloud, ready to run your XCUITests. XCUITest (a framework within the larger XCTest Testing framework) is Apple's automation framework, which comes with Apple's XCode IDE, capable of running automated UI tests on iPhone and iPad devices. ## Setup XCUITest is packaged in a `zip` file. You will need two files to perform XCUITesting on our cloud: - An `.ipa` file of the actual mobile application that you want to test. - Another `.zip` file of the XCUITests. If you just want to give this a try, we have a [TestingBot XCUITest Demo App](https://github.com/testingbot/xcuitest-example-app) which you can use with the following steps. ## TestingBot CLI The easiest way to run XCUITests on TestingBot is using our CLI tool. It handles uploading your app, test ZIP, and running tests in a single command. ### Installation ```bash npm install -g @testingbot/cli ``` ### Authentication Authenticate using this command: ```bash testingbot login ``` This opens your browser for authentication. After logging in, your credentials are saved to `~/.testingbot` and you can start using the CLI. #### Other Authentication Methods - **Command-line options** : `--api-key` and `--api-secret` - **Environment variables** : `TB_KEY` and `TB_SECRET` - **Config file** : Create `~/.testingbot` with content `key:secret` ### Quick Start Run XCUITests with a single command: ```bash testingbot xcuitest app.ipa app-test.zip --device "iPhone 16" --real-device ``` The CLI will: 1. Upload your app IPA to TestingBot 2. Upload your test ZIP 3. Start the test run 4. Show real-time progress and results ## Upload iOS App You can upload your `.ipa` file to TestingBot storage with our [REST-API](https://testingbot.com/support/api). [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) When using the CLI, the app upload is handled automatically as part of the `testingbot xcuitest` command. Simply specify your app IPA file as the first argument: ```bash testingbot xcuitest /path/to/app/sample-debug.ipa /path/to/test/app-test.zip --device "iPhone 16" --real-device ``` Alternatively, you can use the `--app` option: ```bash testingbot xcuitest --app /path/to/app/sample-debug.ipa --test-app /path/to/test/app-test.zip --device "iPhone 16" --real-device ``` ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/xcuitest/app" \ -F "file=@/path/to/app/file/sample-debug.ipa" ``` The API response will include an `id` response, which you can use in the other API calls. ```json { "id": 4012 } ``` ## Upload XCUITest Suite If you do not have a `.zip` file yet, please see our [Build Testsuite](https://testingbot.com/support/app-automate/xcuitest/build-testsuite) documentation. [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) When using the CLI, the test ZIP upload is handled automatically. Specify your test ZIP as the second argument: ```bash testingbot xcuitest app.ipa app-test.zip --device "iPhone 16" --real-device ``` Or use the `--test-app` option: ```bash testingbot xcuitest --app app.ipa --test-app app-test.zip --device "iPhone 16" --real-device ``` To upload the XCUITest suite, in `.zip` file, please use our [REST-API](https://testingbot.com/support/api). ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/xcuitest/:id/tests" \ -F "file=@/path/to/app/file/application-debug-test.zip" ``` Replace the `:id` with the identifier you received during the app upload call. (In this example 4012) ## Run XCUI tests Once you've uploaded both your app and test suite, you can start a XCUITest on the TestingBot cloud. [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) Use the CLI to run tests with device selection options: ```bash # Basic usage with device selection testingbot xcuitest app.ipa app-test.zip --device "iPhone 16" --real-device # With test name and build identifier testingbot xcuitest app.ipa app-test.zip \ --device "iPhone 16" \ --real-device \ --name "Smoke Tests" \ --build "v2.1.0" ``` ### Device Options | Option | Description | | --- | --- | | `--device ` | Device name (e.g., "iPhone 16", "iPad.\*") | | `--platform-version ` | iOS version (e.g., "17.0", "18.2") | | `--real-device` | Use a real device instead of simulator | | `--tablet-only` | Only allocate tablet devices (iPads) | | `--phone-only` | Only allocate phone devices (iPhones) | | `--orientation ` | Screen orientation: PORTRAIT or LANDSCAPE | #### Localization & Network ```bash # With localization settings testingbot xcuitest app.ipa app-test.zip \ --device "iPhone 16" \ --real-device \ --locale "DE" \ --language "de" \ --timezone "Europe/Berlin" # With network throttling and geolocation testingbot xcuitest app.ipa app-test.zip \ --device "iPhone 16" \ --real-device \ --throttle-network "3G" \ --geo-location "DE" ``` Specify one or more capabilities, to indicate on which iOS devices you want to run your tests on. ![iOS selected](https://testingbot.com/assets/environments/svg/ios-383468addf160fa18d0e431f529420739d7f7d1206175f682fead627d2e99a52.svg) iOS 18.2 › iPhone 16 Loading environments... Please wait while we load the available browsers and platforms. Replace the `:id` with the identifier you received during the app upload call. (In this example 4012) We offer special parameters which you can use to allocate a device: | Regex Input | Result | | --- | --- | | `"iPhone.*"` | This will allocate any available iPhone device (phone) | | `"*"` | This will allocate a random available device | | `"iPhone (8\|11)"` | This will allocate either an iPhone 8 or 11 | | `"iPhone 6S?$"` | This will allocate either an iPhone 6 or 6S | Some Examples: ```bash # find any iPhone, except 6 or 6S -d '{"capabilities":[{"deviceName":"^iPhone(?! 6S?$).*", "platformName":"iOS", "realDevice": true}]}' # find any device which name starts with iPad -d '{"capabilities":[{"deviceName":"iPad.*", "platformName":"iOS", "realDevice": true}]}' ``` ### Tablet Only You can specify the `tabletOnly` capability when you only want to allocate a tablet device. ### Phone Only You can specify the `phoneOnly` capability when you only want to allocate a phone device. ```bash -d '{"capabilities":[{"deviceName":"*", "tabletOnly": true, "platformName":"iOS", "realDevice": true}]}' -d '{"capabilities":[{"deviceName":"*", "phoneOnly": true, "platformName":"iOS", "realDevice": true}]}' ``` ## View XCUITest Results The XCUITest results will be available in the [TestingBot dashboard](https://testingbot.com/members). Each test result contains a video, test logs and other meta data generated during the test run. [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) By default, the CLI waits for test completion and displays real-time progress: ```bash testingbot xcuitest app.ipa app-test.zip --device "iPhone 16" --real-device ``` The CLI will show: - Upload progress for app and test ZIP - Device allocation status - Live output from XCUITests - Final pass/fail status ### Async Mode Use `--async` to start tests without waiting for results: ```bash testingbot xcuitest app.ipa app-test.zip --device "iPhone 16" --real-device --async ``` #### Download Reports Download a JUnit or HTML report after test completion: ```bash testingbot xcuitest app.ipa app-test.zip --device "iPhone 16" --real-device \ --report junit \ --report-output-dir ./reports ``` You can also use an API call to get the results from the run(s): ```bash curl -u api_key:api_secret \ "https://api.testingbot.com/v1/app-automate/xcuitest/:id" ``` Replace the `:id` with the identifier you received during the app upload call. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/xcuitest/build-testsuite # Building XCUITest To test with XCUITest on TestingBot, you will need to build and export the XCUITests you have created. If you do not have a `zip` file of the XCUITest suite, you can follow the examples below. The `zip` file should contain a folder similar to this: ```bash xcuitest-sampleUITests-Runner.app ``` ## Using XCode (13 and higher) 1. From the XCode schemes dropdown menu, select your app scheme and **Any iOS Device** as device 2. Click **Product -\> Clean** (`Cmd+Shift+K`) 3. Click **Product -\> Build For -\> Testing** (`Cmd+Shift+U`) 4. Go to **Product -\> Show Build Folder in Finder** 5. In Finder, navigate to **Products -\> Debug-iphoneos**. The directory should contain a file that ends with `-Runner.app`. Right click and choose the compress option to generate a `zip` file. ## Using XCode (Lower than 13.0) 1. From the XCode schemes dropdown menu, select your app scheme and **Generic iOS device** as device 2. Click **Product -\> Clean** (`Cmd+Shift+K`) 3. Click **Product -\> Build For -\> Testing** (`Cmd+Shift+U`) 4. Right click the `.app` file using the Project Navigator, and select **Show in Finder** 5. In Finder, the directory should contain a file called `-Runner.app`. Right click and choose the compress option to generate a `zip` file. ## Using Command Line To build the `.zip` file from the command line, please follow these steps: 1. Change to the project directory and build for testing: ```bash xcodebuild -scheme build-for-testing ``` 2. Go to the **DerivedData** directory. To know the location of this directory, open Xcode and go to **Preferences -\> Locations**. 3. Now you can create the zip file. In the **DerivedData** directory, go to the directory containing the name of your app and some random characters. ```bash cd appname-random_characters cd Build/Products cd Debug-iphoneos zip --symlinks -r .zip -Runner.app ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/xcuitest/change-screen-orientation # Change Device Orientation You can change the iOS device's screen orientation before an XCUITest session starts by passing in a `orientation` option. ## Device Orientation Example See the example below on how to change the orientation. You can choose between `PORTRAIT` (default) and `LANDSCAPE`. [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash testingbot xcuitest app.ipa app-test.zip \ --device "iPhone 13" \ --real-device \ --orientation LANDSCAPE ``` The default orientation is set to `PORTRAIT` for all XCUITests. ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/xcuitest/:id/run" \ -d '{ "options": { "orientation": "LANDSCAPE" }, "capabilities":[{"version":"15.5", "deviceName":"iPhone 13", "platformName":"iOS"}]}' \ -H "Content-Type: application/json" ``` The default orientation is set to `PORTRAIT` for all XCUITests. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/xcuitest/instrumentation # XCUITest Instrumentation Logs Instrumentation logs are logs generated by XCUITest. These logs list all the steps that were performed during the test, with any errors or meta data available from the XCUITest runner. These logs are available in the [TestingBot dashboard](https://testingbot.com/members) as well as through the [REST API](https://testingbot.com/support/api). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/xcuitest/simulate-network-conditions # Simulate Network Conditions TestingBot allows you to test your mobile apps under various network conditions, including download speed, upload speed, packet loss and latency. ## Simulate network conditions using a predefined network profile TestingBot provides a set of predefined network profiles which you can choose from. | Profile Name | Bandwidth down/up (kbps) | Packet Loss (%) | Latency (ms) | | --- | --- | --- | --- | | Edge | 250/150 | 0 | 300 | | 3G | 400/100 | 0 | 100 | | 4G | 18000/9000 | 0 | 100 | | airplane | 0 | 100 | 0 | [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash testingbot xcuitest app.ipa app-test.zip \ --device "iPhone 13" \ --real-device \ --throttle-network "3G" ``` ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/xcuitest/:id/run" \ -d '{ "options": { "throttle_network": "3G" }, "capabilities":[{"version":"15.5", "deviceName":"iPhone 13", "platformName":"iOS"}]}' \ -H "Content-Type: application/json" ``` ## Simulate network conditions using a custom network profile You can specify a custom set of parameters, including: - Download rate (kbps) - Upload rate (kbps) - Packet loss (%) - Latency (ms) [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) The CLI supports predefined network profiles. For custom network profiles, use the cURL API directly. ```bash # Predefined profiles: 4G, 3G, Edge, airplane testingbot xcuitest app.ipa app-test.zip \ --device "iPhone 13" \ --real-device \ --throttle-network "Edge" ``` ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/xcuitest/:id/run" \ -d '{ "options": { "throttle_network": { "uploadSpeed": 10240, "downloadSpeed": 10240, "latency": 0, "loss": 0 } }, "capabilities":[{"version":"15.5", "deviceName":"iPhone 13", "platformName":"iOS"}]}' \ -H "Content-Type: application/json" ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/xcuitest/set-localization-options # Set localization options TestingBot provides an option to run XCUITests on a localized version of your iOS mobile application. Configure the language, locale and timezone of the remote iOS device before running your XCUITest suite. ## Setting Language for iOS XCUI Tests Test a localized version of your mobile app with iOS XCUITest, by changing the language of the application under test. You will need to specify a `language` option with the ISO code of the language you want to use. [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash testingbot xcuitest app.ipa app-test.zip \ --device "iPhone 13" \ --real-device \ --language "fr" ``` Use an [ISO 639-1 language code](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) to set the language of your app. Your app needs to be compiled with support for this language to notice a change. ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/xcuitest/:id/run" \ -d '{ "options": { "language": "fr" }, "capabilities":[{"version":"15.5", "deviceName":"iPhone 13", "platformName":"iOS"}]}' \ -H "Content-Type: application/json" ``` Use an [ISO 639-1 language code](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) to set the language of your app. Your app needs to be compiled with support for this language to notice a change. ## Set a locale for XCUI Testing You can change the locale before running your XCUITest with the `locale` parameter. Please use the CA format (country name abbreviation). For example: `DE` for Germany. [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash testingbot xcuitest app.ipa app-test.zip \ --device "iPhone 13" \ --real-device \ --locale "DE" ``` ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/xcuitest/:id/run" \ -d '{ "options": { "locale": "DE" }, "capabilities":[{"version":"15.5", "deviceName":"iPhone 13", "platformName":"iOS"}]}' \ -H "Content-Type: application/json" ``` ## Change timezone for XCUITests If you'd like to change the timezone before starting your XCUITests, you can use the `timeZone` option. Please use a timezone format from the [list of timezones](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones). [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash testingbot xcuitest app.ipa app-test.zip \ --device "iPhone 13" \ --real-device \ --timezone "America/New_York" ``` ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/xcuitest/:id/run" \ -d '{ "options": { "timeZone": "America/New_York" }, "capabilities":[{"version":"15.5", "deviceName":"iPhone 13", "platformName":"iOS"}]}' \ -H "Content-Type: application/json" ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/xcuitest/set-ip-geolocation # Use IP geolocation TestingBot's GeoIP feature allows you to test your app from various parts of the world, by using IP addresses in [over 20 different countries](https://testingbot.com/support/web-automate/selenium/test-options#geo). When you enable GeoIP, traffic originating from the iOS device which is running your XCUITest will come from an IP address in the country you specified. ## GeoIP example To get started with location testing, you can specify the geoLocation option when starting a XCUITest. Please see the example below where we specify that the app should connect through Germany. [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) ```bash testingbot xcuitest app.ipa app-test.zip \ --device "iPhone 13" \ --real-device \ --geo-location "DE" ``` The `DE` abbreviation is the ISO code of Germany. You can find other [GeoIP country codes](https://testingbot.com/support/web-automate/selenium/test-options#geo) to use. ```bash curl -u api_key:api_secret \ -X POST "https://api.testingbot.com/v1/app-automate/xcuitest/:id/run" \ -d '{ "options": { "geoLocation": "DE" }, "capabilities":[{"version":"15.5", "deviceName":"iPhone 13", "platformName":"iOS"}]}' \ -H "Content-Type: application/json" ``` The `DE` abbreviation is the ISO code of Germany. You can find other [GeoIP country codes](https://testingbot.com/support/web-automate/selenium/test-options#geo) to use. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/app-automate/xcuitest/test-reports # XCUITest Reports TestingBot currently offers both test reports from inside the TestingBot member dashboard, or through REST-API where you can use a [JUnit formatted XML report](https://testingbot.com#junit). ## Dashboard Report When you run XCUITests with TestingBot, you will find the test results, together with a video of the test and various other logs, in the [TestingBot member dashboard](https://testingbot.com/members). ## API report You can query the TestingBot API to fetch the result of the XCUITest run(s). [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) By default, the CLI waits for test completion and displays results in real-time: ```bash testingbot xcuitest app.ipa app-test.zip --device "iPhone 16" --real-device ``` The CLI will show: - Upload progress for app and test ZIP - Device allocation status - Live output from XCUITests - Final pass/fail status Use `--async` to start tests without waiting for results: ```bash testingbot xcuitest app.ipa app-test.zip --device "iPhone 16" --real-device --async ``` ```bash curl -u api_key:api_secret \ "https://api.testingbot.com/v1/app-automate/xcuitest/:id/:run_id" ``` Replace the `:id` and `:run_id` with the identifiers you received during the app upload call and the run call. You will get back a response similar to this one: ```json {"id":216,"created_at":"2024-04-04T15:49:47.000Z","status":"DONE","capabilities":{"deviceName":"iPhone .*","platformName":"iOS"},"success":false,"test":{"sessionId":"ffe-4b453ea45667-61a0a28dbf36-171224576223-3e","environment":{"name":"XCUITest","os":"iPhone 15 - 17.0","version":"17.0"}},"version":"1.0"} ``` ## JUnit XML report You can query our API to fetch a JUnit XML report which you can use a post-build step in your CI/CD environment. **Report Structure** In the XML report, each session maps to a `testsuite` tag. Inside that tag, multiple `testcase` items are available, each mapping to a single XCUITest. If a test failed, a `failure` or `error` tag will be available. [CLI](https://testingbot.com#) [cURL](https://testingbot.com#) Download the JUnit XML report after test completion: ```bash testingbot xcuitest app.ipa app-test.zip \ --device "iPhone 16" \ --real-device \ --report junit \ --report-output-dir ./reports ``` You can also download an HTML report: ```bash testingbot xcuitest app.ipa app-test.zip \ --device "iPhone 16" \ --real-device \ --report html \ --report-output-dir ./reports ``` ```bash curl -u api_key:api_secret \ "https://api.testingbot.com/v1/app-automate/xcuitest/:id/report" ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate # Web Automation with TestingBot TestingBot provides a cloud-based platform for running automated tests on web applications, using various popular testing frameworks and tools. ![Selenium Testing](https://testingbot.com/assets/support/selenium-266ecc0599068fc6eca0bcd9174fb46053c95cebe1c66f7a50c480f3f1a52929.svg) ## [Selenium](https://testingbot.com/support/web-automate/selenium) Selenium is a popular test-framework, designed to automate website testing. - [Java](https://testingbot.com/support/web-automate/selenium/java) - [C#/.NET](https://testingbot.com/support/web-automate/selenium/csharp) - [NodeJS](https://testingbot.com/support/web-automate/selenium/nodejs) - [Python](https://testingbot.com/support/web-automate/selenium/python) - [PHP](https://testingbot.com/support/web-automate/selenium/php) - [Ruby](https://testingbot.com/support/web-automate/selenium/ruby) ![Playwright Testing](https://testingbot.com/assets/support/playwright-2dcb4e75b55ac0d1135cfa47be6137b13f46a2047ce78b96a34f51b0e2cc0d96.svg) ## [Playwright](https://testingbot.com/support/web-automate/playwright) Playwright is a NodeJS library to automate Chromium, Firefox and WebKit. - [Java](https://testingbot.com/support/web-automate/playwright/java) - [C#/.NET](https://testingbot.com/support/web-automate/playwright/dotnet) - [Python](https://testingbot.com/support/web-automate/playwright/pytest) - [Jest](https://testingbot.com/support/web-automate/playwright/jest) ![Puppeteer Testing](https://testingbot.com/assets/support/puppeteer-0a8cdd38af69dabd46638afbb3bffc8a372902949aa599663ec379f3e875263c.svg) ## [Puppeteer](https://testingbot.com/support/web-automate/puppeteer) Puppeteer is a NodeJS library to automate Chrome and Firefox browsers. - [Jest](https://testingbot.com/support/web-automate/puppeteer/jest) - [WebdriverIO](https://testingbot.com/support/web-automate/puppeteer/webdriverio) - [CodeceptJS](https://testingbot.com/support/web-automate/puppeteer/codeceptjs) - [Scraping](https://testingbot.com/support/web-automate/puppeteer/scraping) ![Cypress Testing](https://testingbot.com/assets/support/cypress-409c9fd3f79115ed7676aa1df4f6a217dc4402b3366a61984924aec7b4fefa97.svg) ## [Cypress](https://testingbot.com/support/web-automate/cypress) Cypress is a JavaScript-based end-to-end testing framework. - [Tutorial](https://testingbot.com/support/web-automate/cypress/tutorial) ## Others Other automation tools that work with TestingBot's browser grid. - [Vibium](https://testingbot.com/support/web-automate/vibium) --- URL: https://testingbot.com/support/web-automate/headless # Headless Testing Documentation Headless testing **significantly minimizes the time it takes to run a test**. Because the browser runs in headless mode, it is not necessary to render the graphical user interface. This means the browser starts faster and uses less resources, resulting in quicker test execution. You can still take screenshots during headless mode. Video recording of the test however is not available in headless mode. ## Chrome Headless Testing To run headless tests on Chrome, simply set `headless: true` in `tb:options`. This tells TestingBot to launch the browser in headless mode and automatically disables video recording. We recommend using the `LINUX` platform for the fastest headless testing experience. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [WebdriverIO](https://testingbot.com#) [Ruby](https://testingbot.com#) [C#](https://testingbot.com#) [PHP](https://testingbot.com#) [Robot Framework](https://testingbot.com#) ```java import org.openqa.selenium.WebDriver; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URL; import java.util.HashMap; import java.util.Map; public class HeadlessChromeTest { public static void main(String[] args) throws Exception { ChromeOptions chromeOptions = new ChromeOptions(); chromeOptions.setPlatformName("LINUX"); chromeOptions.setBrowserVersion("latest"); Map tbOptions = new HashMap<>(); tbOptions.put("key", "API_KEY"); tbOptions.put("secret", "API_SECRET"); tbOptions.put("name", "Chrome Headless Test"); tbOptions.put("headless", true); chromeOptions.setCapability("tb:options", tbOptions); WebDriver driver = new RemoteWebDriver( new URL("https://hub.testingbot.com/wd/hub"), chromeOptions ); driver.get("https://www.google.com"); System.out.println("Title: " + driver.getTitle()); driver.quit(); } } ``` ```python from selenium import webdriver from selenium.webdriver.chrome.options import Options chrome_options = Options() chrome_options.set_capability("platformName", "LINUX") chrome_options.set_capability("browserVersion", "latest") chrome_options.set_capability("tb:options", { "key": "API_KEY", "secret": "API_SECRET", "name": "Chrome Headless Test", "headless": True }) driver = webdriver.Remote( command_executor="https://hub.testingbot.com/wd/hub", options=chrome_options ) driver.get("https://www.google.com") print("Title:", driver.title) driver.quit() ``` ```javascript const { Builder } = require('selenium-webdriver'); async function runHeadlessTest() { const driver = await new Builder() .forBrowser('chrome') .withCapabilities({ 'browserName': 'chrome', 'browserVersion': 'latest', 'platformName': 'LINUX', 'tb:options': { 'key': 'API_KEY', 'secret': 'API_SECRET', 'name': 'Chrome Headless Test', 'headless': true } }) .usingServer('https://hub.testingbot.com/wd/hub') .build(); try { await driver.get('https://www.google.com'); const title = await driver.getTitle(); console.log('Title:', title); } finally { await driver.quit(); } } runHeadlessTest(); ``` ```javascript // wdio.conf.js exports.config = { user: 'API_KEY', key: 'API_SECRET', capabilities: [{ browserName: 'chrome', browserVersion: 'latest', platformName: 'LINUX', 'tb:options': { name: 'Chrome Headless Test', headless: true } }], services: [['testingbot']], // Your test configuration specs: ['./test/specs/**/*.js'], framework: 'mocha', reporters: ['spec'] }; // test/specs/example.js describe('Headless Chrome Test', () => { it('should load Google', async () => { await browser.url('https://www.google.com'); const title = await browser.getTitle(); console.log('Title:', title); }); }); ``` ```ruby require 'selenium-webdriver' options = Selenium::WebDriver::Chrome::Options.new options.platform_name = 'LINUX' options.browser_version = 'latest' caps = Selenium::WebDriver::Remote::Capabilities.new( 'tb:options': { key: 'API_KEY', secret: 'API_SECRET', name: 'Chrome Headless Test', headless: true } ) driver = Selenium::WebDriver.for( :remote, url: 'https://hub.testingbot.com/wd/hub', capabilities: [options, caps] ) driver.get('https://www.google.com') puts "Title: #{driver.title}" driver.quit ``` ```csharp using OpenQA.Selenium; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Remote; var chromeOptions = new ChromeOptions(); chromeOptions.PlatformName = "LINUX"; chromeOptions.BrowserVersion = "latest"; var tbOptions = new Dictionary { ["key"] = "API_KEY", ["secret"] = "API_SECRET", ["name"] = "Chrome Headless Test", ["headless"] = true }; chromeOptions.AddAdditionalOption("tb:options", tbOptions); IWebDriver driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600) ); driver.Navigate().GoToUrl("https://www.google.com"); Console.WriteLine("Title: " + driver.Title); driver.Quit(); ``` ```php setCapability('browserVersion', 'latest'); $capabilities->setCapability('platformName', 'LINUX'); $capabilities->setCapability('tb:options', [ 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'Chrome Headless Test', 'headless' => true ]); $driver = RemoteWebDriver::create( 'https://hub.testingbot.com/wd/hub', $capabilities, 60000, 60000 ); $driver->get('https://www.google.com'); echo "Title: " . $driver->getTitle() . "\n"; $driver->quit(); ?> ``` Save the example below in a `headless_chrome.robot` file: ```robot ***Settings*** Library SeleniumLibrary Library TestingBot Library Collections Test Setup Open Headless Chrome Browser Test Teardown Close Test Browser ***Variables*** ${TB_KEY} API_KEY ${TB_SECRET} API_SECRET ***Test Cases*** Chrome Headless Test Go To https://www.google.com Title Should Be Google ***Keywords*** Open Headless Chrome Browser ${chrome_options}= Evaluate selenium.webdriver.ChromeOptions() modules=selenium.webdriver # Set W3C capabilities Call Method ${chrome_options} set_capability platformName LINUX Call Method ${chrome_options} set_capability browserVersion latest # Set TestingBot options with headless enabled ${tb_options}= Create Dictionary name=Chrome Headless Test headless=${True} Call Method ${chrome_options} set_capability tb:options ${tb_options} Open Browser about:blank ... remote_url=https://${TB_KEY}:${TB_SECRET}@hub.testingbot.com/wd/hub ... options=${chrome_options} Close Test Browser Report TestingBot Status ... ${SUITE_NAME} | ${TEST_NAME} ... ${TEST_STATUS} ... ${TB_KEY}:${TB_SECRET} Close All Browsers ``` Save the code below in a `TestingBot.py` file: ```python import requests from robot.libraries.BuiltIn import BuiltIn def report_testingbot_status(name, status, credentials): selenium = BuiltIn().get_library_instance('SeleniumLibrary') session_id = selenium.driver.session_id payload = {'test[name]': name, 'test[success]': int(status == 'PASS')} key, secret = credentials.split(':') url = 'https://api.testingbot.com/v1/tests/{0}'.format(session_id) response = requests.put(url, data=payload, auth=(key, secret)) assert response.status_code == 200, response.text ``` To run the test: ```bash pip install robotframework-seleniumlibrary requests PYTHONPATH=$PYTHONPATH:. robot headless_chrome.robot ``` ## Firefox Headless Testing To run headless tests on Firefox, simply set `headless: true` in `tb:options`. This tells TestingBot to launch the browser in headless mode and automatically disables video recording. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [WebdriverIO](https://testingbot.com#) [Ruby](https://testingbot.com#) [C#](https://testingbot.com#) [PHP](https://testingbot.com#) [Robot Framework](https://testingbot.com#) ```java import org.openqa.selenium.WebDriver; import org.openqa.selenium.firefox.FirefoxOptions; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URL; import java.util.HashMap; import java.util.Map; public class HeadlessFirefoxTest { public static void main(String[] args) throws Exception { FirefoxOptions firefoxOptions = new FirefoxOptions(); firefoxOptions.setPlatformName("LINUX"); firefoxOptions.setBrowserVersion("latest"); Map tbOptions = new HashMap<>(); tbOptions.put("key", "API_KEY"); tbOptions.put("secret", "API_SECRET"); tbOptions.put("name", "Firefox Headless Test"); tbOptions.put("headless", true); firefoxOptions.setCapability("tb:options", tbOptions); WebDriver driver = new RemoteWebDriver( new URL("https://hub.testingbot.com/wd/hub"), firefoxOptions ); driver.get("https://www.google.com"); System.out.println("Title: " + driver.getTitle()); driver.quit(); } } ``` ```python from selenium import webdriver from selenium.webdriver.firefox.options import Options firefox_options = Options() firefox_options.set_capability("platformName", "LINUX") firefox_options.set_capability("browserVersion", "latest") firefox_options.set_capability("tb:options", { "key": "API_KEY", "secret": "API_SECRET", "name": "Firefox Headless Test", "headless": True }) driver = webdriver.Remote( command_executor="https://hub.testingbot.com/wd/hub", options=firefox_options ) driver.get("https://www.google.com") print("Title:", driver.title) driver.quit() ``` ```javascript const { Builder } = require('selenium-webdriver'); async function runHeadlessTest() { const driver = await new Builder() .forBrowser('firefox') .withCapabilities({ 'browserName': 'firefox', 'browserVersion': 'latest', 'platformName': 'LINUX', 'tb:options': { 'key': 'API_KEY', 'secret': 'API_SECRET', 'name': 'Firefox Headless Test', 'headless': true } }) .usingServer('https://hub.testingbot.com/wd/hub') .build(); try { await driver.get('https://www.google.com'); const title = await driver.getTitle(); console.log('Title:', title); } finally { await driver.quit(); } } runHeadlessTest(); ``` ```javascript // wdio.conf.js exports.config = { user: 'API_KEY', key: 'API_SECRET', capabilities: [{ browserName: 'firefox', browserVersion: 'latest', platformName: 'LINUX', 'tb:options': { name: 'Firefox Headless Test', headless: true } }], services: [['testingbot']], // Your test configuration specs: ['./test/specs/**/*.js'], framework: 'mocha', reporters: ['spec'] }; // test/specs/example.js describe('Headless Firefox Test', () => { it('should load Google', async () => { await browser.url('https://www.google.com'); const title = await browser.getTitle(); console.log('Title:', title); }); }); ``` ```ruby require 'selenium-webdriver' options = Selenium::WebDriver::Firefox::Options.new options.platform_name = 'LINUX' options.browser_version = 'latest' caps = Selenium::WebDriver::Remote::Capabilities.new( 'tb:options': { key: 'API_KEY', secret: 'API_SECRET', name: 'Firefox Headless Test', headless: true } ) driver = Selenium::WebDriver.for( :remote, url: 'https://hub.testingbot.com/wd/hub', capabilities: [options, caps] ) driver.get('https://www.google.com') puts "Title: #{driver.title}" driver.quit ``` ```csharp using OpenQA.Selenium; using OpenQA.Selenium.Firefox; using OpenQA.Selenium.Remote; var firefoxOptions = new FirefoxOptions(); firefoxOptions.PlatformName = "LINUX"; firefoxOptions.BrowserVersion = "latest"; var tbOptions = new Dictionary { ["key"] = "API_KEY", ["secret"] = "API_SECRET", ["name"] = "Firefox Headless Test", ["headless"] = true }; firefoxOptions.AddAdditionalOption("tb:options", tbOptions); IWebDriver driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), firefoxOptions.ToCapabilities(), TimeSpan.FromSeconds(600) ); driver.Navigate().GoToUrl("https://www.google.com"); Console.WriteLine("Title: " + driver.Title); driver.Quit(); ``` ```php setCapability('browserVersion', 'latest'); $capabilities->setCapability('platformName', 'LINUX'); $capabilities->setCapability('tb:options', [ 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'Firefox Headless Test', 'headless' => true ]); $driver = RemoteWebDriver::create( 'https://hub.testingbot.com/wd/hub', $capabilities, 60000, 60000 ); $driver->get('https://www.google.com'); echo "Title: " . $driver->getTitle() . "\n"; $driver->quit(); ?> ``` Save the example below in a `headless_firefox.robot` file: ```robot ***Settings*** Library SeleniumLibrary Library TestingBot Library Collections Test Setup Open Headless Firefox Browser Test Teardown Close Test Browser ***Variables*** ${TB_KEY} API_KEY ${TB_SECRET} API_SECRET ***Test Cases*** Firefox Headless Test Go To https://www.google.com Title Should Be Google ***Keywords*** Open Headless Firefox Browser ${firefox_options}= Evaluate selenium.webdriver.FirefoxOptions() modules=selenium.webdriver # Set W3C capabilities Call Method ${firefox_options} set_capability platformName LINUX Call Method ${firefox_options} set_capability browserVersion latest # Set TestingBot options with headless enabled ${tb_options}= Create Dictionary name=Firefox Headless Test headless=${True} Call Method ${firefox_options} set_capability tb:options ${tb_options} Open Browser about:blank ... remote_url=https://${TB_KEY}:${TB_SECRET}@hub.testingbot.com/wd/hub ... options=${firefox_options} Close Test Browser Report TestingBot Status ... ${SUITE_NAME} | ${TEST_NAME} ... ${TEST_STATUS} ... ${TB_KEY}:${TB_SECRET} Close All Browsers ``` Save the code below in a `TestingBot.py` file: ```python import requests from robot.libraries.BuiltIn import BuiltIn def report_testingbot_status(name, status, credentials): selenium = BuiltIn().get_library_instance('SeleniumLibrary') session_id = selenium.driver.session_id payload = {'test[name]': name, 'test[success]': int(status == 'PASS')} key, secret = credentials.split(':') url = 'https://api.testingbot.com/v1/tests/{0}'.format(session_id) response = requests.put(url, data=payload, auth=(key, secret)) assert response.status_code == 200, response.text ``` To run the test: ```bash pip install robotframework-seleniumlibrary requests PYTHONPATH=$PYTHONPATH:. robot headless_firefox.robot ``` ## Microsoft Edge Headless Testing To run headless tests on Microsoft Edge, simply set `headless: true` in `tb:options`. This tells TestingBot to launch the browser in headless mode and automatically disables video recording. We recommend using the `WIN11` platform for Edge headless testing. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [WebdriverIO](https://testingbot.com#) [Ruby](https://testingbot.com#) [C#](https://testingbot.com#) [PHP](https://testingbot.com#) [Robot Framework](https://testingbot.com#) ```java import org.openqa.selenium.WebDriver; import org.openqa.selenium.edge.EdgeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URL; import java.util.HashMap; import java.util.Map; public class HeadlessEdgeTest { public static void main(String[] args) throws Exception { EdgeOptions edgeOptions = new EdgeOptions(); edgeOptions.setPlatformName("WIN11"); edgeOptions.setBrowserVersion("latest"); Map tbOptions = new HashMap<>(); tbOptions.put("key", "API_KEY"); tbOptions.put("secret", "API_SECRET"); tbOptions.put("name", "Edge Headless Test"); tbOptions.put("headless", true); edgeOptions.setCapability("tb:options", tbOptions); WebDriver driver = new RemoteWebDriver( new URL("https://hub.testingbot.com/wd/hub"), edgeOptions ); driver.get("https://www.google.com"); System.out.println("Title: " + driver.getTitle()); driver.quit(); } } ``` ```python from selenium import webdriver from selenium.webdriver.edge.options import Options edge_options = Options() edge_options.set_capability("platformName", "WIN11") edge_options.set_capability("browserVersion", "latest") edge_options.set_capability("tb:options", { "key": "API_KEY", "secret": "API_SECRET", "name": "Edge Headless Test", "headless": True }) driver = webdriver.Remote( command_executor="https://hub.testingbot.com/wd/hub", options=edge_options ) driver.get("https://www.google.com") print("Title:", driver.title) driver.quit() ``` ```javascript const { Builder } = require('selenium-webdriver'); async function runHeadlessTest() { const driver = await new Builder() .forBrowser('MicrosoftEdge') .withCapabilities({ 'browserName': 'MicrosoftEdge', 'browserVersion': 'latest', 'platformName': 'WIN11', 'tb:options': { 'key': 'API_KEY', 'secret': 'API_SECRET', 'name': 'Edge Headless Test', 'headless': true } }) .usingServer('https://hub.testingbot.com/wd/hub') .build(); try { await driver.get('https://www.google.com'); const title = await driver.getTitle(); console.log('Title:', title); } finally { await driver.quit(); } } runHeadlessTest(); ``` ```javascript // wdio.conf.js exports.config = { user: 'API_KEY', key: 'API_SECRET', capabilities: [{ browserName: 'MicrosoftEdge', browserVersion: 'latest', platformName: 'WIN11', 'tb:options': { name: 'Edge Headless Test', headless: true } }], services: [['testingbot']], // Your test configuration specs: ['./test/specs/**/*.js'], framework: 'mocha', reporters: ['spec'] }; // test/specs/example.js describe('Headless Edge Test', () => { it('should load Google', async () => { await browser.url('https://www.google.com'); const title = await browser.getTitle(); console.log('Title:', title); }); }); ``` ```ruby require 'selenium-webdriver' options = Selenium::WebDriver::Edge::Options.new options.platform_name = 'WIN11' options.browser_version = 'latest' caps = Selenium::WebDriver::Remote::Capabilities.new( 'tb:options': { key: 'API_KEY', secret: 'API_SECRET', name: 'Edge Headless Test', headless: true } ) driver = Selenium::WebDriver.for( :remote, url: 'https://hub.testingbot.com/wd/hub', capabilities: [options, caps] ) driver.get('https://www.google.com') puts "Title: #{driver.title}" driver.quit ``` ```csharp using OpenQA.Selenium; using OpenQA.Selenium.Edge; using OpenQA.Selenium.Remote; var edgeOptions = new EdgeOptions(); edgeOptions.PlatformName = "WIN11"; edgeOptions.BrowserVersion = "latest"; var tbOptions = new Dictionary { ["key"] = "API_KEY", ["secret"] = "API_SECRET", ["name"] = "Edge Headless Test", ["headless"] = true }; edgeOptions.AddAdditionalOption("tb:options", tbOptions); IWebDriver driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), edgeOptions.ToCapabilities(), TimeSpan.FromSeconds(600) ); driver.Navigate().GoToUrl("https://www.google.com"); Console.WriteLine("Title: " + driver.Title); driver.Quit(); ``` ```php setCapability('browserVersion', 'latest'); $capabilities->setCapability('platformName', 'WIN11'); $capabilities->setCapability('tb:options', [ 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'Edge Headless Test', 'headless' => true ]); $driver = RemoteWebDriver::create( 'https://hub.testingbot.com/wd/hub', $capabilities, 60000, 60000 ); $driver->get('https://www.google.com'); echo "Title: " . $driver->getTitle() . "\n"; $driver->quit(); ?> ``` Save the example below in a `headless_edge.robot` file: ```robot ***Settings*** Library SeleniumLibrary Library TestingBot Library Collections Test Setup Open Headless Edge Browser Test Teardown Close Test Browser ***Variables*** ${TB_KEY} API_KEY ${TB_SECRET} API_SECRET ***Test Cases*** Edge Headless Test Go To https://www.google.com Title Should Be Google ***Keywords*** Open Headless Edge Browser ${edge_options}= Evaluate selenium.webdriver.EdgeOptions() modules=selenium.webdriver # Set W3C capabilities Call Method ${edge_options} set_capability platformName WIN11 Call Method ${edge_options} set_capability browserVersion latest # Set TestingBot options with headless enabled ${tb_options}= Create Dictionary name=Edge Headless Test headless=${True} Call Method ${edge_options} set_capability tb:options ${tb_options} Open Browser about:blank ... remote_url=https://${TB_KEY}:${TB_SECRET}@hub.testingbot.com/wd/hub ... options=${edge_options} Close Test Browser Report TestingBot Status ... ${SUITE_NAME} | ${TEST_NAME} ... ${TEST_STATUS} ... ${TB_KEY}:${TB_SECRET} Close All Browsers ``` Save the code below in a `TestingBot.py` file: ```python import requests from robot.libraries.BuiltIn import BuiltIn def report_testingbot_status(name, status, credentials): selenium = BuiltIn().get_library_instance('SeleniumLibrary') session_id = selenium.driver.session_id payload = {'test[name]': name, 'test[success]': int(status == 'PASS')} key, secret = credentials.split(':') url = 'https://api.testingbot.com/v1/tests/{0}'.format(session_id) response = requests.put(url, data=payload, auth=(key, secret)) assert response.status_code == 200, response.text ``` To run the test: ```bash pip install robotframework-seleniumlibrary requests PYTHONPATH=$PYTHONPATH:. robot headless_edge.robot ``` ## Safari Headless Testing **Safari does not support headless mode.** Apple has not implemented headless functionality in Safari or the SafariDriver. If you need to run Safari tests, they will always run in headed (visible) mode. For headless testing requirements, consider using Chrome, Firefox or Edge as alternatives. At TestingBot we run all Safari 17 and higher versions on Apple Silicon for maximum performance. ## Mobile Headless Testing TestingBot uses Appium to run mobile tests. Appium supports running your iOS and Android tests in headless mode (no screen rendering). To enable headless mode for mobile testing, specify the `isHeadless` capability in your test configuration. ### iOS Headless Testing [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [C#](https://testingbot.com#) ```java import io.appium.java_client.ios.IOSDriver; import io.appium.java_client.ios.options.XCUITestOptions; import java.net.URL; import java.util.HashMap; import java.util.Map; public class HeadlessIOSTest { public static void main(String[] args) throws Exception { XCUITestOptions options = new XCUITestOptions(); options.setPlatformName("iOS"); options.setPlatformVersion("18.0"); options.setDeviceName("iPhone 16"); options.setBrowserName("safari"); options.setCapability("appium:isHeadless", true); Map tbOptions = new HashMap<>(); tbOptions.put("key", "API_KEY"); tbOptions.put("secret", "API_SECRET"); tbOptions.put("name", "iOS Headless Test"); options.setCapability("tb:options", tbOptions); IOSDriver driver = new IOSDriver( new URL("https://hub.testingbot.com/wd/hub"), options ); driver.get("https://www.google.com"); System.out.println("Title: " + driver.getTitle()); driver.quit(); } } ``` ```python from appium import webdriver from appium.options.ios import XCUITestOptions options = XCUITestOptions() options.platform_name = "iOS" options.platform_version = "18.0" options.device_name = "iPhone 16" options.browser_name = "safari" options.set_capability("appium:isHeadless", True) options.set_capability("tb:options", { "key": "API_KEY", "secret": "API_SECRET", "name": "iOS Headless Test" }) driver = webdriver.Remote( command_executor="https://hub.testingbot.com/wd/hub", options=options ) driver.get("https://www.google.com") print("Title:", driver.title) driver.quit() ``` ```javascript const { remote } = require('webdriverio'); async function runHeadlessIOSTest() { const driver = await remote({ hostname: 'hub.testingbot.com', port: 443, path: '/wd/hub', protocol: 'https', capabilities: { 'platformName': 'iOS', 'appium:platformVersion': '18.0', 'appium:deviceName': 'iPhone 16', 'browserName': 'safari', 'appium:isHeadless': true, 'tb:options': { 'key': 'API_KEY', 'secret': 'API_SECRET', 'name': 'iOS Headless Test' } } }); await driver.url('https://www.google.com'); const title = await driver.getTitle(); console.log('Title:', title); await driver.deleteSession(); } runHeadlessIOSTest(); ``` ```ruby require 'appium_lib' caps = { platformName: 'iOS', 'appium:platformVersion': '18.0', 'appium:deviceName': 'iPhone 16', browserName: 'safari', 'appium:isHeadless': true, 'tb:options': { key: 'API_KEY', secret: 'API_SECRET', name: 'iOS Headless Test' } } opts = { caps: caps, appium_lib: { server_url: 'https://hub.testingbot.com/wd/hub' } } driver = Appium::Driver.new(opts, true) driver.start_driver driver.get('https://www.google.com') puts "Title: #{driver.title}" driver.quit ``` ```csharp using OpenQA.Selenium.Appium; using OpenQA.Selenium.Appium.iOS; var options = new AppiumOptions(); options.PlatformName = "iOS"; options.AddAdditionalAppiumOption("appium:platformVersion", "18.0"); options.AddAdditionalAppiumOption("appium:deviceName", "iPhone 16"); options.AddAdditionalAppiumOption("browserName", "safari"); options.AddAdditionalAppiumOption("appium:isHeadless", true); var tbOptions = new Dictionary { ["key"] = "API_KEY", ["secret"] = "API_SECRET", ["name"] = "iOS Headless Test" }; options.AddAdditionalAppiumOption("tb:options", tbOptions); var driver = new IOSDriver( new Uri("https://hub.testingbot.com/wd/hub"), options, TimeSpan.FromSeconds(600) ); driver.Navigate().GoToUrl("https://www.google.com"); Console.WriteLine("Title: " + driver.Title); driver.Quit(); ``` ### Android Headless Testing [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [C#](https://testingbot.com#) ```java import io.appium.java_client.android.AndroidDriver; import io.appium.java_client.android.options.UiAutomator2Options; import java.net.URL; import java.util.HashMap; import java.util.Map; public class HeadlessAndroidTest { public static void main(String[] args) throws Exception { UiAutomator2Options options = new UiAutomator2Options(); options.setPlatformName("Android"); options.setPlatformVersion("14.0"); options.setDeviceName("Pixel 8"); options.setBrowserName("chrome"); options.setCapability("appium:isHeadless", true); Map tbOptions = new HashMap<>(); tbOptions.put("key", "API_KEY"); tbOptions.put("secret", "API_SECRET"); tbOptions.put("name", "Android Headless Test"); options.setCapability("tb:options", tbOptions); AndroidDriver driver = new AndroidDriver( new URL("https://hub.testingbot.com/wd/hub"), options ); driver.get("https://www.google.com"); System.out.println("Title: " + driver.getTitle()); driver.quit(); } } ``` ```python from appium import webdriver from appium.options.android import UiAutomator2Options options = UiAutomator2Options() options.platform_name = "Android" options.platform_version = "14.0" options.device_name = "Pixel 8" options.browser_name = "chrome" options.set_capability("appium:isHeadless", True) options.set_capability("tb:options", { "key": "API_KEY", "secret": "API_SECRET", "name": "Android Headless Test" }) driver = webdriver.Remote( command_executor="https://hub.testingbot.com/wd/hub", options=options ) driver.get("https://www.google.com") print("Title:", driver.title) driver.quit() ``` ```javascript const { remote } = require('webdriverio'); async function runHeadlessAndroidTest() { const driver = await remote({ hostname: 'hub.testingbot.com', port: 443, path: '/wd/hub', protocol: 'https', capabilities: { 'platformName': 'Android', 'appium:platformVersion': '14.0', 'appium:deviceName': 'Pixel 8', 'browserName': 'chrome', 'appium:isHeadless': true, 'tb:options': { 'key': 'API_KEY', 'secret': 'API_SECRET', 'name': 'Android Headless Test' } } }); await driver.url('https://www.google.com'); const title = await driver.getTitle(); console.log('Title:', title); await driver.deleteSession(); } runHeadlessAndroidTest(); ``` ```ruby require 'appium_lib' caps = { platformName: 'Android', 'appium:platformVersion': '14.0', 'appium:deviceName': 'Pixel 8', browserName: 'chrome', 'appium:isHeadless': true, 'tb:options': { key: 'API_KEY', secret: 'API_SECRET', name: 'Android Headless Test' } } opts = { caps: caps, appium_lib: { server_url: 'https://hub.testingbot.com/wd/hub' } } driver = Appium::Driver.new(opts, true) driver.start_driver driver.get('https://www.google.com') puts "Title: #{driver.title}" driver.quit ``` ```csharp using OpenQA.Selenium.Appium; using OpenQA.Selenium.Appium.Android; var options = new AppiumOptions(); options.PlatformName = "Android"; options.AddAdditionalAppiumOption("appium:platformVersion", "14.0"); options.AddAdditionalAppiumOption("appium:deviceName", "Pixel 8"); options.AddAdditionalAppiumOption("browserName", "chrome"); options.AddAdditionalAppiumOption("appium:isHeadless", true); var tbOptions = new Dictionary { ["key"] = "API_KEY", ["secret"] = "API_SECRET", ["name"] = "Android Headless Test" }; options.AddAdditionalAppiumOption("tb:options", tbOptions); var driver = new AndroidDriver( new Uri("https://hub.testingbot.com/wd/hub"), options, TimeSpan.FromSeconds(600) ); driver.Navigate().GoToUrl("https://www.google.com"); Console.WriteLine("Title: " + driver.Title); driver.Quit(); ``` ## Benefits of Headless Testing - **Faster Test Execution:** Without rendering the UI, tests start and complete more quickly. - **Lower Resource Usage:** Headless browsers consume less memory and CPU. - **Ideal for CI/CD:** Perfect for automated pipelines where visual output is not needed. - **Screenshots Supported:** You can still capture screenshots during headless tests. ## Limitations - **No Video Recording:** Video recording is not available during headless tests. - **Debugging:** Debugging can be more challenging without seeing the browser UI. For debugging purposes, we recommend switching to headed mode temporarily to visually inspect the browser during test execution. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/browsers - [Desktop Browsers & Emulators](https://testingbot.com#) - [Physical Mobile Devices](https://testingbot.com/support/app-automate/devices) # List of available browsers Below is a list of **6783 combinations** available for testing. Click a combination to see how to run an automated or manual test. ## Test Configuration [Run Manual Test](https://testingbot.com/members/manual/new) Java Python Ruby NodeJS C# Jump to: | [Win 11](https://testingbot.com#os-win11) [Win 10](https://testingbot.com#os-win10) [Win 8.1](https://testingbot.com#os-win81) [Win 8](https://testingbot.com#os-win8) [Win 7](https://testingbot.com#os-win7) | [Golden Gate](https://testingbot.com#os-goldengate) [Tahoe](https://testingbot.com#os-tahoe) [Sequoia](https://testingbot.com#os-sequoia) [Sonoma](https://testingbot.com#os-sonoma) [Ventura](https://testingbot.com#os-ventura) [Monterey](https://testingbot.com#os-monterey) [Big Sur](https://testingbot.com#os-bigsur) [Catalina](https://testingbot.com#os-catalina) [Mojave](https://testingbot.com#os-mojave) [High Sierra](https://testingbot.com#os-highsierra) | [Linux](https://testingbot.com#os-linux) | [iOS](https://testingbot.com#os-ios) [Android](https://testingbot.com#os-android) ![Windows 11 Browser Testing](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) ## Windows 11 (64-bit) ![googlechrome](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg "Googlechrome") ### Chrome [beta](https://testingbot.com# "Chrome") [dev](https://testingbot.com# "Chrome") [153](https://testingbot.com# "Chrome 153") [152](https://testingbot.com# "Chrome 152") [151](https://testingbot.com# "Chrome 151") [150](https://testingbot.com# "Chrome 150") [149](https://testingbot.com# "Chrome 149") [148](https://testingbot.com# "Chrome 148") [147](https://testingbot.com# "Chrome 147") [146](https://testingbot.com# "Chrome 146") [145](https://testingbot.com# "Chrome 145") [144](https://testingbot.com# "Chrome 144") [143](https://testingbot.com# "Chrome 143") [142](https://testingbot.com# "Chrome 142") [141](https://testingbot.com# "Chrome 141") [140](https://testingbot.com# "Chrome 140") [139](https://testingbot.com# "Chrome 139") [138](https://testingbot.com# "Chrome 138") [137](https://testingbot.com# "Chrome 137") [136](https://testingbot.com# "Chrome 136") [135](https://testingbot.com# "Chrome 135") [134](https://testingbot.com# "Chrome 134") [133](https://testingbot.com# "Chrome 133") [132](https://testingbot.com# "Chrome 132") [131](https://testingbot.com# "Chrome 131") [130](https://testingbot.com# "Chrome 130") [129](https://testingbot.com# "Chrome 129") [128](https://testingbot.com# "Chrome 128") [127](https://testingbot.com# "Chrome 127") [126](https://testingbot.com# "Chrome 126") [125](https://testingbot.com# "Chrome 125") [124](https://testingbot.com# "Chrome 124") [123](https://testingbot.com# "Chrome 123") [122](https://testingbot.com# "Chrome 122") [121](https://testingbot.com# "Chrome 121") [120](https://testingbot.com# "Chrome 120") [119](https://testingbot.com# "Chrome 119") [118](https://testingbot.com# "Chrome 118") [117](https://testingbot.com# "Chrome 117") [116](https://testingbot.com# "Chrome 116") [115](https://testingbot.com# "Chrome 115") [114](https://testingbot.com# "Chrome 114") [113](https://testingbot.com# "Chrome 113") [112](https://testingbot.com# "Chrome 112") [111](https://testingbot.com# "Chrome 111") [110](https://testingbot.com# "Chrome 110") [109](https://testingbot.com# "Chrome 109") [108](https://testingbot.com# "Chrome 108") [107](https://testingbot.com# "Chrome 107") [106](https://testingbot.com# "Chrome 106") [105](https://testingbot.com# "Chrome 105") [104](https://testingbot.com# "Chrome 104") [103](https://testingbot.com# "Chrome 103") [102](https://testingbot.com# "Chrome 102") [101](https://testingbot.com# "Chrome 101") [100](https://testingbot.com# "Chrome 100") [99](https://testingbot.com# "Chrome 99") [98](https://testingbot.com# "Chrome 98") [97](https://testingbot.com# "Chrome 97") [96](https://testingbot.com# "Chrome 96") [95](https://testingbot.com# "Chrome 95") [94](https://testingbot.com# "Chrome 94") [93](https://testingbot.com# "Chrome 93") [92](https://testingbot.com# "Chrome 92") [91](https://testingbot.com# "Chrome 91") [90](https://testingbot.com# "Chrome 90") [89](https://testingbot.com# "Chrome 89") [88](https://testingbot.com# "Chrome 88") [87](https://testingbot.com# "Chrome 87") [86](https://testingbot.com# "Chrome 86") [85](https://testingbot.com# "Chrome 85") [84](https://testingbot.com# "Chrome 84") [83](https://testingbot.com# "Chrome 83") [81](https://testingbot.com# "Chrome 81") [80](https://testingbot.com# "Chrome 80") [79](https://testingbot.com# "Chrome 79") [78](https://testingbot.com# "Chrome 78") [77](https://testingbot.com# "Chrome 77") [76](https://testingbot.com# "Chrome 76") [75](https://testingbot.com# "Chrome 75") [74](https://testingbot.com# "Chrome 74") [73](https://testingbot.com# "Chrome 73") [72](https://testingbot.com# "Chrome 72") [71](https://testingbot.com# "Chrome 71") [70](https://testingbot.com# "Chrome 70") [69](https://testingbot.com# "Chrome 69") [68](https://testingbot.com# "Chrome 68") [67](https://testingbot.com# "Chrome 67") [66](https://testingbot.com# "Chrome 66") [65](https://testingbot.com# "Chrome 65") [64](https://testingbot.com# "Chrome 64") [63](https://testingbot.com# "Chrome 63") [62](https://testingbot.com# "Chrome 62") [61](https://testingbot.com# "Chrome 61") [60](https://testingbot.com# "Chrome 60") [59](https://testingbot.com# "Chrome 59") [58](https://testingbot.com# "Chrome 58") [57](https://testingbot.com# "Chrome 57") [56](https://testingbot.com# "Chrome 56") [55](https://testingbot.com# "Chrome 55") [54](https://testingbot.com# "Chrome 54") [53](https://testingbot.com# "Chrome 53") [52](https://testingbot.com# "Chrome 52") [51](https://testingbot.com# "Chrome 51") [50](https://testingbot.com# "Chrome 50") [49](https://testingbot.com# "Chrome 49") [48](https://testingbot.com# "Chrome 48") [47](https://testingbot.com# "Chrome 47") [46](https://testingbot.com# "Chrome 46") [45](https://testingbot.com# "Chrome 45") [44](https://testingbot.com# "Chrome 44") [43](https://testingbot.com# "Chrome 43") [42](https://testingbot.com# "Chrome 42") [41](https://testingbot.com# "Chrome 41") [40](https://testingbot.com# "Chrome 40") [39](https://testingbot.com# "Chrome 39") [38](https://testingbot.com# "Chrome 38") [37](https://testingbot.com# "Chrome 37") [36](https://testingbot.com# "Chrome 36") ![firefox](https://testingbot.com/assets/environments/svg/firefox-90dce8de549728e54bd3209cfacb9f1a868c2c8e70d243eb5b930d69cbb2fb1e.svg "Firefox") ### Firefox [dev](https://testingbot.com# "FF dev") [beta](https://testingbot.com# "FF beta") [155](https://testingbot.com# "FF 155") [154](https://testingbot.com# "FF 154") [153](https://testingbot.com# "FF 153") [152](https://testingbot.com# "FF 152") [151](https://testingbot.com# "FF 151") [150](https://testingbot.com# "FF 150") [149](https://testingbot.com# "FF 149") [148](https://testingbot.com# "FF 148") [147](https://testingbot.com# "FF 147") [146](https://testingbot.com# "FF 146") [145](https://testingbot.com# "FF 145") [144](https://testingbot.com# "FF 144") [143](https://testingbot.com# "FF 143") [142](https://testingbot.com# "FF 142") [141](https://testingbot.com# "FF 141") [140](https://testingbot.com# "FF 140") [139](https://testingbot.com# "FF 139") [138](https://testingbot.com# "FF 138") [137](https://testingbot.com# "FF 137") [136](https://testingbot.com# "FF 136") [135](https://testingbot.com# "FF 135") [134](https://testingbot.com# "FF 134") [133](https://testingbot.com# "FF 133") [132](https://testingbot.com# "FF 132") [131](https://testingbot.com# "FF 131") [130](https://testingbot.com# "FF 130") [129](https://testingbot.com# "FF 129") [128](https://testingbot.com# "FF 128") [127](https://testingbot.com# "FF 127") [126](https://testingbot.com# "FF 126") [125](https://testingbot.com# "FF 125") [124](https://testingbot.com# "FF 124") [123](https://testingbot.com# "FF 123") [122](https://testingbot.com# "FF 122") [121](https://testingbot.com# "FF 121") [120](https://testingbot.com# "FF 120") [119](https://testingbot.com# "FF 119") [118](https://testingbot.com# "FF 118") [117](https://testingbot.com# "FF 117") [116](https://testingbot.com# "FF 116") [115](https://testingbot.com# "FF 115") [114](https://testingbot.com# "FF 114") [113](https://testingbot.com# "FF 113") [112](https://testingbot.com# "FF 112") [111](https://testingbot.com# "FF 111") [110](https://testingbot.com# "FF 110") [109](https://testingbot.com# "FF 109") [108](https://testingbot.com# "FF 108") [107](https://testingbot.com# "FF 107") [106](https://testingbot.com# "FF 106") [105](https://testingbot.com# "FF 105") [104](https://testingbot.com# "FF 104") [103](https://testingbot.com# "FF 103") [102](https://testingbot.com# "FF 102") [101](https://testingbot.com# "FF 101") [100](https://testingbot.com# "FF 100") [99](https://testingbot.com# "FF 99") [98](https://testingbot.com# "FF 98") [97](https://testingbot.com# "FF 97") [96](https://testingbot.com# "FF 96") [95](https://testingbot.com# "FF 95") [94](https://testingbot.com# "FF 94") [93](https://testingbot.com# "FF 93") [92](https://testingbot.com# "FF 92") [91](https://testingbot.com# "FF 91") [90](https://testingbot.com# "FF 90") [89](https://testingbot.com# "FF 89") [88](https://testingbot.com# "FF 88") [87](https://testingbot.com# "FF 87") [86](https://testingbot.com# "FF 86") [85](https://testingbot.com# "FF 85") [84](https://testingbot.com# "FF 84") [83](https://testingbot.com# "FF 83") [82](https://testingbot.com# "FF 82") [81](https://testingbot.com# "FF 81") [80](https://testingbot.com# "FF 80") [79](https://testingbot.com# "FF 79") [78](https://testingbot.com# "FF 78") [77](https://testingbot.com# "FF 77") [76](https://testingbot.com# "FF 76") [75](https://testingbot.com# "FF 75") [74](https://testingbot.com# "FF 74") [73](https://testingbot.com# "FF 73") [72](https://testingbot.com# "FF 72") [71](https://testingbot.com# "FF 71") [70](https://testingbot.com# "FF 70") [69](https://testingbot.com# "FF 69") [68](https://testingbot.com# "FF 68") [67](https://testingbot.com# "FF 67") [66](https://testingbot.com# "FF 66") [65](https://testingbot.com# "FF 65") [64](https://testingbot.com# "FF 64") [63](https://testingbot.com# "FF 63") [62](https://testingbot.com# "FF 62") [61](https://testingbot.com# "FF 61") [60](https://testingbot.com# "FF 60") [59](https://testingbot.com# "FF 59") [58](https://testingbot.com# "FF 58") [57](https://testingbot.com# "FF 57") [56](https://testingbot.com# "FF 56") [55](https://testingbot.com# "FF 55") [54](https://testingbot.com# "FF 54") [53](https://testingbot.com# "FF 53") [52](https://testingbot.com# "FF 52") [51](https://testingbot.com# "FF 51") [50](https://testingbot.com# "FF 50") [49](https://testingbot.com# "FF 49") [48](https://testingbot.com# "FF 48") [47](https://testingbot.com# "FF 47") [46](https://testingbot.com# "FF 46") [45](https://testingbot.com# "FF 45") [44](https://testingbot.com# "FF 44") [43](https://testingbot.com# "FF 43") [42](https://testingbot.com# "FF 42") [41](https://testingbot.com# "FF 41") [40](https://testingbot.com# "FF 40") [39](https://testingbot.com# "FF 39") [38](https://testingbot.com# "FF 38") [37](https://testingbot.com# "FF 37") [36](https://testingbot.com# "FF 36") [35](https://testingbot.com# "FF 35") [34](https://testingbot.com# "FF 34") [33](https://testingbot.com# "FF 33") [32](https://testingbot.com# "FF 32") [31](https://testingbot.com# "FF 31") [30](https://testingbot.com# "FF 30") [29](https://testingbot.com# "FF 29") [28](https://testingbot.com# "FF 28") [27](https://testingbot.com# "FF 27") [26](https://testingbot.com# "FF 26") [25](https://testingbot.com# "FF 25") [24](https://testingbot.com# "FF 24") [23](https://testingbot.com# "FF 23") [22](https://testingbot.com# "FF 22") [21](https://testingbot.com# "FF 21") [20](https://testingbot.com# "FF 20") [19](https://testingbot.com# "FF 19") [18](https://testingbot.com# "FF 18") [17](https://testingbot.com# "FF 17") [16](https://testingbot.com# "FF 16") [15](https://testingbot.com# "FF 15") [14](https://testingbot.com# "FF 14") [13](https://testingbot.com# "FF 13") [12](https://testingbot.com# "FF 12") [11](https://testingbot.com# "FF 11") [10](https://testingbot.com# "FF 10") [9](https://testingbot.com# "FF 9") [8](https://testingbot.com# "FF 8") [7](https://testingbot.com# "FF 7") [6](https://testingbot.com# "FF 6") [5](https://testingbot.com# "FF 5") [4](https://testingbot.com# "FF 4") ![microsoftedge](https://testingbot.com/assets/environments/svg/edge-b639cac0c7a843271a0d3a6ccf894cc296064b63b2cd2920209bc5d030f3c70b.svg "Microsoftedge") ### Edge [dev](https://testingbot.com# "Edge") [beta](https://testingbot.com# "Edge") [153](https://testingbot.com# "Edge 153") [152](https://testingbot.com# "Edge 152") [151](https://testingbot.com# "Edge 151") [150](https://testingbot.com# "Edge 150") [149](https://testingbot.com# "Edge 149") [148](https://testingbot.com# "Edge 148") [147](https://testingbot.com# "Edge 147") [146](https://testingbot.com# "Edge 146") [145](https://testingbot.com# "Edge 145") [144](https://testingbot.com# "Edge 144") [143](https://testingbot.com# "Edge 143") [142](https://testingbot.com# "Edge 142") [141](https://testingbot.com# "Edge 141") [140](https://testingbot.com# "Edge 140") [139](https://testingbot.com# "Edge 139") [138](https://testingbot.com# "Edge 138") [137](https://testingbot.com# "Edge 137") [136](https://testingbot.com# "Edge 136") [135](https://testingbot.com# "Edge 135") [134](https://testingbot.com# "Edge 134") [133](https://testingbot.com# "Edge 133") [132](https://testingbot.com# "Edge 132") [131](https://testingbot.com# "Edge 131") [130](https://testingbot.com# "Edge 130") [129](https://testingbot.com# "Edge 129") [128](https://testingbot.com# "Edge 128") [127](https://testingbot.com# "Edge 127") [126](https://testingbot.com# "Edge 126") [125](https://testingbot.com# "Edge 125") [124](https://testingbot.com# "Edge 124") [123](https://testingbot.com# "Edge 123") [122](https://testingbot.com# "Edge 122") [121](https://testingbot.com# "Edge 121") [120](https://testingbot.com# "Edge 120") [119](https://testingbot.com# "Edge 119") [118](https://testingbot.com# "Edge 118") [117](https://testingbot.com# "Edge 117") [116](https://testingbot.com# "Edge 116") [115](https://testingbot.com# "Edge 115") [114](https://testingbot.com# "Edge 114") [113](https://testingbot.com# "Edge 113") [112](https://testingbot.com# "Edge 112") [111](https://testingbot.com# "Edge 111") [110](https://testingbot.com# "Edge 110") [109](https://testingbot.com# "Edge 109") [108](https://testingbot.com# "Edge 108") [107](https://testingbot.com# "Edge 107") [106](https://testingbot.com# "Edge 106") [105](https://testingbot.com# "Edge 105") [104](https://testingbot.com# "Edge 104") [103](https://testingbot.com# "Edge 103") [102](https://testingbot.com# "Edge 102") [101](https://testingbot.com# "Edge 101") [100](https://testingbot.com# "Edge 100") [99](https://testingbot.com# "Edge 99") [98](https://testingbot.com# "Edge 98") [97](https://testingbot.com# "Edge 97") [96](https://testingbot.com# "Edge 96") [95](https://testingbot.com# "Edge 95") [94](https://testingbot.com# "Edge 94") [93](https://testingbot.com# "Edge 93") [92](https://testingbot.com# "Edge 92") [91](https://testingbot.com# "Edge 91") [90](https://testingbot.com# "Edge 90") [89](https://testingbot.com# "Edge 89") [88](https://testingbot.com# "Edge 88") [87](https://testingbot.com# "Edge 87") [86](https://testingbot.com# "Edge 86") [85](https://testingbot.com# "Edge 85") [84](https://testingbot.com# "Edge 84") [83](https://testingbot.com# "Edge 83") [81](https://testingbot.com# "Edge 81") [80](https://testingbot.com# "Edge 80") [79](https://testingbot.com# "Edge 79") ![opera](https://testingbot.com/assets/environments/svg/opera-fe6e1d716472033125a969dfe6489ab45f0eded91e9260292d6c4ce9b682a330.svg "Opera") ### Opera [135](https://testingbot.com# "Opera 135") [134](https://testingbot.com# "Opera 134") [130](https://testingbot.com# "Opera 130") [129](https://testingbot.com# "Opera 129") [128](https://testingbot.com# "Opera 128") [127](https://testingbot.com# "Opera 127") [126](https://testingbot.com# "Opera 126") [125](https://testingbot.com# "Opera 125") [124](https://testingbot.com# "Opera 124") [123](https://testingbot.com# "Opera 123") [122](https://testingbot.com# "Opera 122") [121](https://testingbot.com# "Opera 121") [120](https://testingbot.com# "Opera 120") [119](https://testingbot.com# "Opera 119") [118](https://testingbot.com# "Opera 118") [117](https://testingbot.com# "Opera 117") [116](https://testingbot.com# "Opera 116") [115](https://testingbot.com# "Opera 115") [114](https://testingbot.com# "Opera 114") [113](https://testingbot.com# "Opera 113") [112](https://testingbot.com# "Opera 112") [111](https://testingbot.com# "Opera 111") [110](https://testingbot.com# "Opera 110") [109](https://testingbot.com# "Opera 109") [108](https://testingbot.com# "Opera 108") [107](https://testingbot.com# "Opera 107") [106](https://testingbot.com# "Opera 106") [105](https://testingbot.com# "Opera 105") [104](https://testingbot.com# "Opera 104") [103](https://testingbot.com# "Opera 103") [102](https://testingbot.com# "Opera 102") [101](https://testingbot.com# "Opera 101") [100](https://testingbot.com# "Opera 100") [99](https://testingbot.com# "Opera 99") [98](https://testingbot.com# "Opera 98") [97](https://testingbot.com# "Opera 97") [96](https://testingbot.com# "Opera 96") [95](https://testingbot.com# "Opera 95") [94](https://testingbot.com# "Opera 94") [93](https://testingbot.com# "Opera 93") [92](https://testingbot.com# "Opera 92") [91](https://testingbot.com# "Opera 91") [90](https://testingbot.com# "Opera 90") [89](https://testingbot.com# "Opera 89") [88](https://testingbot.com# "Opera 88") [87](https://testingbot.com# "Opera 87") [86](https://testingbot.com# "Opera 86") [85](https://testingbot.com# "Opera 85") [84](https://testingbot.com# "Opera 84") [83](https://testingbot.com# "Opera 83") [82](https://testingbot.com# "Opera 82") [81](https://testingbot.com# "Opera 81") [80](https://testingbot.com# "Opera 80") [79](https://testingbot.com# "Opera 79") [78](https://testingbot.com# "Opera 78") [77](https://testingbot.com# "Opera 77") [76](https://testingbot.com# "Opera 76") [75](https://testingbot.com# "Opera 75") [74](https://testingbot.com# "Opera 74") [73](https://testingbot.com# "Opera 73") [72](https://testingbot.com# "Opera 72") [71](https://testingbot.com# "Opera 71") [70](https://testingbot.com# "Opera 70") [69](https://testingbot.com# "Opera 69") [68](https://testingbot.com# "Opera 68") [67](https://testingbot.com# "Opera 67") [66](https://testingbot.com# "Opera 66") [65](https://testingbot.com# "Opera 65") [64](https://testingbot.com# "Opera 64") [63](https://testingbot.com# "Opera 63") [62](https://testingbot.com# "Opera 62") [60](https://testingbot.com# "Opera 60") [59](https://testingbot.com# "Opera 59") [58](https://testingbot.com# "Opera 58") [57](https://testingbot.com# "Opera 57") [56](https://testingbot.com# "Opera 56") [55](https://testingbot.com# "Opera 55") [54](https://testingbot.com# "Opera 54") [53](https://testingbot.com# "Opera 53") [52](https://testingbot.com# "Opera 52") [51](https://testingbot.com# "Opera 51") [50](https://testingbot.com# "Opera 50") ![Windows 10 Browser Testing](https://testingbot.com/assets/environments/svg/windows10-f19b2b5055d9a272b1c8b35711154fc9cca1ae14442b48771e44680d28faffee.svg) ## Windows 10 (64-bit) ![googlechrome](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg "Googlechrome") ### Chrome [beta](https://testingbot.com# "Chrome") [dev](https://testingbot.com# "Chrome") [153](https://testingbot.com# "Chrome 153") [152](https://testingbot.com# "Chrome 152") [151](https://testingbot.com# "Chrome 151") [150](https://testingbot.com# "Chrome 150") [149](https://testingbot.com# "Chrome 149") [148](https://testingbot.com# "Chrome 148") [147](https://testingbot.com# "Chrome 147") [146](https://testingbot.com# "Chrome 146") [145](https://testingbot.com# "Chrome 145") [144](https://testingbot.com# "Chrome 144") [143](https://testingbot.com# "Chrome 143") [142](https://testingbot.com# "Chrome 142") [141](https://testingbot.com# "Chrome 141") [140](https://testingbot.com# "Chrome 140") [139](https://testingbot.com# "Chrome 139") [138](https://testingbot.com# "Chrome 138") [137](https://testingbot.com# "Chrome 137") [136](https://testingbot.com# "Chrome 136") [135](https://testingbot.com# "Chrome 135") [134](https://testingbot.com# "Chrome 134") [133](https://testingbot.com# "Chrome 133") [132](https://testingbot.com# "Chrome 132") [131](https://testingbot.com# "Chrome 131") [130](https://testingbot.com# "Chrome 130") [129](https://testingbot.com# "Chrome 129") [128](https://testingbot.com# "Chrome 128") [127](https://testingbot.com# "Chrome 127") [126](https://testingbot.com# "Chrome 126") [125](https://testingbot.com# "Chrome 125") [124](https://testingbot.com# "Chrome 124") [123](https://testingbot.com# "Chrome 123") [122](https://testingbot.com# "Chrome 122") [121](https://testingbot.com# "Chrome 121") [120](https://testingbot.com# "Chrome 120") [119](https://testingbot.com# "Chrome 119") [118](https://testingbot.com# "Chrome 118") [117](https://testingbot.com# "Chrome 117") [116](https://testingbot.com# "Chrome 116") [115](https://testingbot.com# "Chrome 115") [114](https://testingbot.com# "Chrome 114") [113](https://testingbot.com# "Chrome 113") [112](https://testingbot.com# "Chrome 112") [111](https://testingbot.com# "Chrome 111") [110](https://testingbot.com# "Chrome 110") [109](https://testingbot.com# "Chrome 109") [108](https://testingbot.com# "Chrome 108") [107](https://testingbot.com# "Chrome 107") [106](https://testingbot.com# "Chrome 106") [105](https://testingbot.com# "Chrome 105") [104](https://testingbot.com# "Chrome 104") [103](https://testingbot.com# "Chrome 103") [102](https://testingbot.com# "Chrome 102") [101](https://testingbot.com# "Chrome 101") [100](https://testingbot.com# "Chrome 100") [99](https://testingbot.com# "Chrome 99") [98](https://testingbot.com# "Chrome 98") [97](https://testingbot.com# "Chrome 97") [96](https://testingbot.com# "Chrome 96") [95](https://testingbot.com# "Chrome 95") [94](https://testingbot.com# "Chrome 94") [93](https://testingbot.com# "Chrome 93") [92](https://testingbot.com# "Chrome 92") [91](https://testingbot.com# "Chrome 91") [90](https://testingbot.com# "Chrome 90") [89](https://testingbot.com# "Chrome 89") [88](https://testingbot.com# "Chrome 88") [87](https://testingbot.com# "Chrome 87") [86](https://testingbot.com# "Chrome 86") [85](https://testingbot.com# "Chrome 85") [84](https://testingbot.com# "Chrome 84") [83](https://testingbot.com# "Chrome 83") [81](https://testingbot.com# "Chrome 81") [80](https://testingbot.com# "Chrome 80") [79](https://testingbot.com# "Chrome 79") [78](https://testingbot.com# "Chrome 78") [77](https://testingbot.com# "Chrome 77") [76](https://testingbot.com# "Chrome 76") [75](https://testingbot.com# "Chrome 75") [74](https://testingbot.com# "Chrome 74") [73](https://testingbot.com# "Chrome 73") [72](https://testingbot.com# "Chrome 72") [71](https://testingbot.com# "Chrome 71") [70](https://testingbot.com# "Chrome 70") [69](https://testingbot.com# "Chrome 69") [68](https://testingbot.com# "Chrome 68") [67](https://testingbot.com# "Chrome 67") [66](https://testingbot.com# "Chrome 66") [65](https://testingbot.com# "Chrome 65") [64](https://testingbot.com# "Chrome 64") [63](https://testingbot.com# "Chrome 63") [62](https://testingbot.com# "Chrome 62") [61](https://testingbot.com# "Chrome 61") [60](https://testingbot.com# "Chrome 60") [59](https://testingbot.com# "Chrome 59") [58](https://testingbot.com# "Chrome 58") [57](https://testingbot.com# "Chrome 57") [56](https://testingbot.com# "Chrome 56") [55](https://testingbot.com# "Chrome 55") [54](https://testingbot.com# "Chrome 54") [53](https://testingbot.com# "Chrome 53") [52](https://testingbot.com# "Chrome 52") [51](https://testingbot.com# "Chrome 51") [50](https://testingbot.com# "Chrome 50") [49](https://testingbot.com# "Chrome 49") [48](https://testingbot.com# "Chrome 48") [47](https://testingbot.com# "Chrome 47") [46](https://testingbot.com# "Chrome 46") [45](https://testingbot.com# "Chrome 45") [44](https://testingbot.com# "Chrome 44") [43](https://testingbot.com# "Chrome 43") [42](https://testingbot.com# "Chrome 42") [41](https://testingbot.com# "Chrome 41") [40](https://testingbot.com# "Chrome 40") [39](https://testingbot.com# "Chrome 39") [38](https://testingbot.com# "Chrome 38") [37](https://testingbot.com# "Chrome 37") [36](https://testingbot.com# "Chrome 36") ![firefox](https://testingbot.com/assets/environments/svg/firefox-90dce8de549728e54bd3209cfacb9f1a868c2c8e70d243eb5b930d69cbb2fb1e.svg "Firefox") ### Firefox [dev](https://testingbot.com# "FF dev") [beta](https://testingbot.com# "FF beta") [155](https://testingbot.com# "FF 155") [154](https://testingbot.com# "FF 154") [153](https://testingbot.com# "FF 153") [152](https://testingbot.com# "FF 152") [151](https://testingbot.com# "FF 151") [150](https://testingbot.com# "FF 150") [149](https://testingbot.com# "FF 149") [148](https://testingbot.com# "FF 148") [147](https://testingbot.com# "FF 147") [146](https://testingbot.com# "FF 146") [145](https://testingbot.com# "FF 145") [144](https://testingbot.com# "FF 144") [143](https://testingbot.com# "FF 143") [142](https://testingbot.com# "FF 142") [141](https://testingbot.com# "FF 141") [140](https://testingbot.com# "FF 140") [139](https://testingbot.com# "FF 139") [138](https://testingbot.com# "FF 138") [137](https://testingbot.com# "FF 137") [136](https://testingbot.com# "FF 136") [135](https://testingbot.com# "FF 135") [134](https://testingbot.com# "FF 134") [133](https://testingbot.com# "FF 133") [132](https://testingbot.com# "FF 132") [131](https://testingbot.com# "FF 131") [130](https://testingbot.com# "FF 130") [129](https://testingbot.com# "FF 129") [128](https://testingbot.com# "FF 128") [127](https://testingbot.com# "FF 127") [126](https://testingbot.com# "FF 126") [125](https://testingbot.com# "FF 125") [124](https://testingbot.com# "FF 124") [123](https://testingbot.com# "FF 123") [122](https://testingbot.com# "FF 122") [121](https://testingbot.com# "FF 121") [120](https://testingbot.com# "FF 120") [119](https://testingbot.com# "FF 119") [118](https://testingbot.com# "FF 118") [117](https://testingbot.com# "FF 117") [116](https://testingbot.com# "FF 116") [115](https://testingbot.com# "FF 115") [114](https://testingbot.com# "FF 114") [113](https://testingbot.com# "FF 113") [112](https://testingbot.com# "FF 112") [111](https://testingbot.com# "FF 111") [110](https://testingbot.com# "FF 110") [109](https://testingbot.com# "FF 109") [108](https://testingbot.com# "FF 108") [107](https://testingbot.com# "FF 107") [106](https://testingbot.com# "FF 106") [105](https://testingbot.com# "FF 105") [104](https://testingbot.com# "FF 104") [103](https://testingbot.com# "FF 103") [102](https://testingbot.com# "FF 102") [101](https://testingbot.com# "FF 101") [100](https://testingbot.com# "FF 100") [99](https://testingbot.com# "FF 99") [98](https://testingbot.com# "FF 98") [97](https://testingbot.com# "FF 97") [96](https://testingbot.com# "FF 96") [95](https://testingbot.com# "FF 95") [94](https://testingbot.com# "FF 94") [93](https://testingbot.com# "FF 93") [92](https://testingbot.com# "FF 92") [91](https://testingbot.com# "FF 91") [90](https://testingbot.com# "FF 90") [89](https://testingbot.com# "FF 89") [88](https://testingbot.com# "FF 88") [87](https://testingbot.com# "FF 87") [86](https://testingbot.com# "FF 86") [85](https://testingbot.com# "FF 85") [84](https://testingbot.com# "FF 84") [83](https://testingbot.com# "FF 83") [82](https://testingbot.com# "FF 82") [81](https://testingbot.com# "FF 81") [80](https://testingbot.com# "FF 80") [79](https://testingbot.com# "FF 79") [78](https://testingbot.com# "FF 78") [77](https://testingbot.com# "FF 77") [76](https://testingbot.com# "FF 76") [75](https://testingbot.com# "FF 75") [74](https://testingbot.com# "FF 74") [73](https://testingbot.com# "FF 73") [72](https://testingbot.com# "FF 72") [71](https://testingbot.com# "FF 71") [70](https://testingbot.com# "FF 70") [69](https://testingbot.com# "FF 69") [68](https://testingbot.com# "FF 68") [67](https://testingbot.com# "FF 67") [66](https://testingbot.com# "FF 66") [65](https://testingbot.com# "FF 65") [64](https://testingbot.com# "FF 64") [63](https://testingbot.com# "FF 63") [62](https://testingbot.com# "FF 62") [61](https://testingbot.com# "FF 61") [60](https://testingbot.com# "FF 60") [59](https://testingbot.com# "FF 59") [58](https://testingbot.com# "FF 58") [57](https://testingbot.com# "FF 57") [56](https://testingbot.com# "FF 56") [55](https://testingbot.com# "FF 55") [54](https://testingbot.com# "FF 54") [53](https://testingbot.com# "FF 53") [52](https://testingbot.com# "FF 52") [51](https://testingbot.com# "FF 51") [50](https://testingbot.com# "FF 50") [49](https://testingbot.com# "FF 49") [48](https://testingbot.com# "FF 48") [47](https://testingbot.com# "FF 47") [46](https://testingbot.com# "FF 46") [45](https://testingbot.com# "FF 45") [44](https://testingbot.com# "FF 44") [43](https://testingbot.com# "FF 43") [42](https://testingbot.com# "FF 42") [41](https://testingbot.com# "FF 41") [40](https://testingbot.com# "FF 40") [39](https://testingbot.com# "FF 39") [38](https://testingbot.com# "FF 38") [37](https://testingbot.com# "FF 37") [36](https://testingbot.com# "FF 36") [35](https://testingbot.com# "FF 35") [34](https://testingbot.com# "FF 34") [33](https://testingbot.com# "FF 33") [32](https://testingbot.com# "FF 32") [31](https://testingbot.com# "FF 31") [30](https://testingbot.com# "FF 30") [29](https://testingbot.com# "FF 29") [28](https://testingbot.com# "FF 28") [27](https://testingbot.com# "FF 27") [26](https://testingbot.com# "FF 26") [25](https://testingbot.com# "FF 25") [24](https://testingbot.com# "FF 24") [23](https://testingbot.com# "FF 23") [22](https://testingbot.com# "FF 22") [21](https://testingbot.com# "FF 21") [20](https://testingbot.com# "FF 20") [19](https://testingbot.com# "FF 19") [18](https://testingbot.com# "FF 18") [17](https://testingbot.com# "FF 17") [16](https://testingbot.com# "FF 16") [15](https://testingbot.com# "FF 15") [14](https://testingbot.com# "FF 14") [13](https://testingbot.com# "FF 13") [12](https://testingbot.com# "FF 12") [11](https://testingbot.com# "FF 11") [10](https://testingbot.com# "FF 10") [9](https://testingbot.com# "FF 9") [8](https://testingbot.com# "FF 8") [7](https://testingbot.com# "FF 7") [6](https://testingbot.com# "FF 6") [5](https://testingbot.com# "FF 5") [4](https://testingbot.com# "FF 4") ![iexplore](https://testingbot.com/assets/environments/svg/ie-075955285f8005808f3d26947aace877f96e5843a33bd4d6e997709d3a294451.svg "Iexplore") ### IE [11](https://testingbot.com# "IE11") ![microsoftedge](https://testingbot.com/assets/environments/svg/edge-b639cac0c7a843271a0d3a6ccf894cc296064b63b2cd2920209bc5d030f3c70b.svg "Microsoftedge") ### Edge [dev](https://testingbot.com# "Edge") [beta](https://testingbot.com# "Edge") [153](https://testingbot.com# "Edge 153") [152](https://testingbot.com# "Edge 152") [151](https://testingbot.com# "Edge 151") [150](https://testingbot.com# "Edge 150") [149](https://testingbot.com# "Edge 149") [148](https://testingbot.com# "Edge 148") [147](https://testingbot.com# "Edge 147") [146](https://testingbot.com# "Edge 146") [145](https://testingbot.com# "Edge 145") [144](https://testingbot.com# "Edge 144") [143](https://testingbot.com# "Edge 143") [142](https://testingbot.com# "Edge 142") [141](https://testingbot.com# "Edge 141") [140](https://testingbot.com# "Edge 140") [139](https://testingbot.com# "Edge 139") [138](https://testingbot.com# "Edge 138") [137](https://testingbot.com# "Edge 137") [136](https://testingbot.com# "Edge 136") [135](https://testingbot.com# "Edge 135") [134](https://testingbot.com# "Edge 134") [133](https://testingbot.com# "Edge 133") [132](https://testingbot.com# "Edge 132") [131](https://testingbot.com# "Edge 131") [130](https://testingbot.com# "Edge 130") [129](https://testingbot.com# "Edge 129") [128](https://testingbot.com# "Edge 128") [127](https://testingbot.com# "Edge 127") [126](https://testingbot.com# "Edge 126") [125](https://testingbot.com# "Edge 125") [124](https://testingbot.com# "Edge 124") [123](https://testingbot.com# "Edge 123") [122](https://testingbot.com# "Edge 122") [121](https://testingbot.com# "Edge 121") [120](https://testingbot.com# "Edge 120") [119](https://testingbot.com# "Edge 119") [118](https://testingbot.com# "Edge 118") [117](https://testingbot.com# "Edge 117") [116](https://testingbot.com# "Edge 116") [115](https://testingbot.com# "Edge 115") [114](https://testingbot.com# "Edge 114") [113](https://testingbot.com# "Edge 113") [112](https://testingbot.com# "Edge 112") [111](https://testingbot.com# "Edge 111") [110](https://testingbot.com# "Edge 110") [109](https://testingbot.com# "Edge 109") [108](https://testingbot.com# "Edge 108") [107](https://testingbot.com# "Edge 107") [106](https://testingbot.com# "Edge 106") [105](https://testingbot.com# "Edge 105") [104](https://testingbot.com# "Edge 104") [103](https://testingbot.com# "Edge 103") [102](https://testingbot.com# "Edge 102") [101](https://testingbot.com# "Edge 101") [100](https://testingbot.com# "Edge 100") [99](https://testingbot.com# "Edge 99") [98](https://testingbot.com# "Edge 98") [97](https://testingbot.com# "Edge 97") [96](https://testingbot.com# "Edge 96") [95](https://testingbot.com# "Edge 95") [94](https://testingbot.com# "Edge 94") [93](https://testingbot.com# "Edge 93") [92](https://testingbot.com# "Edge 92") [91](https://testingbot.com# "Edge 91") [90](https://testingbot.com# "Edge 90") [89](https://testingbot.com# "Edge 89") [88](https://testingbot.com# "Edge 88") [87](https://testingbot.com# "Edge 87") [86](https://testingbot.com# "Edge 86") [85](https://testingbot.com# "Edge 85") [84](https://testingbot.com# "Edge 84") [83](https://testingbot.com# "Edge 83") [81](https://testingbot.com# "Edge 81") [80](https://testingbot.com# "Edge 80") [79](https://testingbot.com# "Edge 79") ![opera](https://testingbot.com/assets/environments/svg/opera-fe6e1d716472033125a969dfe6489ab45f0eded91e9260292d6c4ce9b682a330.svg "Opera") ### Opera [135](https://testingbot.com# "Opera 135") [134](https://testingbot.com# "Opera 134") [130](https://testingbot.com# "Opera 130") [129](https://testingbot.com# "Opera 129") [128](https://testingbot.com# "Opera 128") [127](https://testingbot.com# "Opera 127") [126](https://testingbot.com# "Opera 126") [125](https://testingbot.com# "Opera 125") [124](https://testingbot.com# "Opera 124") [123](https://testingbot.com# "Opera 123") [122](https://testingbot.com# "Opera 122") [121](https://testingbot.com# "Opera 121") [120](https://testingbot.com# "Opera 120") [119](https://testingbot.com# "Opera 119") [118](https://testingbot.com# "Opera 118") [117](https://testingbot.com# "Opera 117") [116](https://testingbot.com# "Opera 116") [115](https://testingbot.com# "Opera 115") [114](https://testingbot.com# "Opera 114") [113](https://testingbot.com# "Opera 113") [112](https://testingbot.com# "Opera 112") [111](https://testingbot.com# "Opera 111") [110](https://testingbot.com# "Opera 110") [109](https://testingbot.com# "Opera 109") [108](https://testingbot.com# "Opera 108") [107](https://testingbot.com# "Opera 107") [106](https://testingbot.com# "Opera 106") [105](https://testingbot.com# "Opera 105") [104](https://testingbot.com# "Opera 104") [103](https://testingbot.com# "Opera 103") [102](https://testingbot.com# "Opera 102") [101](https://testingbot.com# "Opera 101") [100](https://testingbot.com# "Opera 100") [99](https://testingbot.com# "Opera 99") [98](https://testingbot.com# "Opera 98") [97](https://testingbot.com# "Opera 97") [96](https://testingbot.com# "Opera 96") [95](https://testingbot.com# "Opera 95") [94](https://testingbot.com# "Opera 94") [93](https://testingbot.com# "Opera 93") [92](https://testingbot.com# "Opera 92") [91](https://testingbot.com# "Opera 91") [90](https://testingbot.com# "Opera 90") [89](https://testingbot.com# "Opera 89") [88](https://testingbot.com# "Opera 88") [87](https://testingbot.com# "Opera 87") [86](https://testingbot.com# "Opera 86") [85](https://testingbot.com# "Opera 85") [84](https://testingbot.com# "Opera 84") [83](https://testingbot.com# "Opera 83") [82](https://testingbot.com# "Opera 82") [81](https://testingbot.com# "Opera 81") [80](https://testingbot.com# "Opera 80") [79](https://testingbot.com# "Opera 79") [78](https://testingbot.com# "Opera 78") [77](https://testingbot.com# "Opera 77") [76](https://testingbot.com# "Opera 76") [75](https://testingbot.com# "Opera 75") [74](https://testingbot.com# "Opera 74") [73](https://testingbot.com# "Opera 73") [72](https://testingbot.com# "Opera 72") [71](https://testingbot.com# "Opera 71") [70](https://testingbot.com# "Opera 70") [69](https://testingbot.com# "Opera 69") [68](https://testingbot.com# "Opera 68") [67](https://testingbot.com# "Opera 67") [66](https://testingbot.com# "Opera 66") [65](https://testingbot.com# "Opera 65") [64](https://testingbot.com# "Opera 64") [63](https://testingbot.com# "Opera 63") [62](https://testingbot.com# "Opera 62") [60](https://testingbot.com# "Opera 60") [59](https://testingbot.com# "Opera 59") [58](https://testingbot.com# "Opera 58") [57](https://testingbot.com# "Opera 57") [56](https://testingbot.com# "Opera 56") [55](https://testingbot.com# "Opera 55") [54](https://testingbot.com# "Opera 54") [53](https://testingbot.com# "Opera 53") [52](https://testingbot.com# "Opera 52") [51](https://testingbot.com# "Opera 51") [50](https://testingbot.com# "Opera 50") ![Windows 8 Browser Testing](https://testingbot.com/assets/environments/svg/windows10-f19b2b5055d9a272b1c8b35711154fc9cca1ae14442b48771e44680d28faffee.svg) ## Windows 8.1 (64-bit) ![googlechrome](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg "Googlechrome") ### Chrome [109](https://testingbot.com# "Chrome 109") [108](https://testingbot.com# "Chrome 108") [107](https://testingbot.com# "Chrome 107") [106](https://testingbot.com# "Chrome 106") [105](https://testingbot.com# "Chrome 105") [104](https://testingbot.com# "Chrome 104") [103](https://testingbot.com# "Chrome 103") [102](https://testingbot.com# "Chrome 102") [101](https://testingbot.com# "Chrome 101") [100](https://testingbot.com# "Chrome 100") [99](https://testingbot.com# "Chrome 99") [98](https://testingbot.com# "Chrome 98") [97](https://testingbot.com# "Chrome 97") [96](https://testingbot.com# "Chrome 96") [95](https://testingbot.com# "Chrome 95") [94](https://testingbot.com# "Chrome 94") [93](https://testingbot.com# "Chrome 93") [92](https://testingbot.com# "Chrome 92") [91](https://testingbot.com# "Chrome 91") [90](https://testingbot.com# "Chrome 90") [89](https://testingbot.com# "Chrome 89") [88](https://testingbot.com# "Chrome 88") [87](https://testingbot.com# "Chrome 87") [86](https://testingbot.com# "Chrome 86") [85](https://testingbot.com# "Chrome 85") [84](https://testingbot.com# "Chrome 84") [83](https://testingbot.com# "Chrome 83") [81](https://testingbot.com# "Chrome 81") [80](https://testingbot.com# "Chrome 80") [79](https://testingbot.com# "Chrome 79") [78](https://testingbot.com# "Chrome 78") [77](https://testingbot.com# "Chrome 77") [76](https://testingbot.com# "Chrome 76") [75](https://testingbot.com# "Chrome 75") [74](https://testingbot.com# "Chrome 74") [73](https://testingbot.com# "Chrome 73") [72](https://testingbot.com# "Chrome 72") [71](https://testingbot.com# "Chrome 71") [70](https://testingbot.com# "Chrome 70") [69](https://testingbot.com# "Chrome 69") [68](https://testingbot.com# "Chrome 68") [67](https://testingbot.com# "Chrome 67") [66](https://testingbot.com# "Chrome 66") [65](https://testingbot.com# "Chrome 65") [64](https://testingbot.com# "Chrome 64") [63](https://testingbot.com# "Chrome 63") [62](https://testingbot.com# "Chrome 62") [61](https://testingbot.com# "Chrome 61") [60](https://testingbot.com# "Chrome 60") [59](https://testingbot.com# "Chrome 59") [58](https://testingbot.com# "Chrome 58") [57](https://testingbot.com# "Chrome 57") [56](https://testingbot.com# "Chrome 56") [55](https://testingbot.com# "Chrome 55") [54](https://testingbot.com# "Chrome 54") [53](https://testingbot.com# "Chrome 53") [52](https://testingbot.com# "Chrome 52") [51](https://testingbot.com# "Chrome 51") [50](https://testingbot.com# "Chrome 50") [49](https://testingbot.com# "Chrome 49") [48](https://testingbot.com# "Chrome 48") [47](https://testingbot.com# "Chrome 47") [46](https://testingbot.com# "Chrome 46") [45](https://testingbot.com# "Chrome 45") [44](https://testingbot.com# "Chrome 44") [43](https://testingbot.com# "Chrome 43") [42](https://testingbot.com# "Chrome 42") [41](https://testingbot.com# "Chrome 41") [40](https://testingbot.com# "Chrome 40") [39](https://testingbot.com# "Chrome 39") [38](https://testingbot.com# "Chrome 38") [37](https://testingbot.com# "Chrome 37") [36](https://testingbot.com# "Chrome 36") ![firefox](https://testingbot.com/assets/environments/svg/firefox-90dce8de549728e54bd3209cfacb9f1a868c2c8e70d243eb5b930d69cbb2fb1e.svg "Firefox") ### Firefox [115](https://testingbot.com# "FF 115") [114](https://testingbot.com# "FF 114") [113](https://testingbot.com# "FF 113") [112](https://testingbot.com# "FF 112") [111](https://testingbot.com# "FF 111") [110](https://testingbot.com# "FF 110") [109](https://testingbot.com# "FF 109") [108](https://testingbot.com# "FF 108") [107](https://testingbot.com# "FF 107") [106](https://testingbot.com# "FF 106") [105](https://testingbot.com# "FF 105") [104](https://testingbot.com# "FF 104") [103](https://testingbot.com# "FF 103") [102](https://testingbot.com# "FF 102") [101](https://testingbot.com# "FF 101") [100](https://testingbot.com# "FF 100") [99](https://testingbot.com# "FF 99") [98](https://testingbot.com# "FF 98") [97](https://testingbot.com# "FF 97") [96](https://testingbot.com# "FF 96") [95](https://testingbot.com# "FF 95") [94](https://testingbot.com# "FF 94") [93](https://testingbot.com# "FF 93") [92](https://testingbot.com# "FF 92") [91](https://testingbot.com# "FF 91") [90](https://testingbot.com# "FF 90") [89](https://testingbot.com# "FF 89") [88](https://testingbot.com# "FF 88") [87](https://testingbot.com# "FF 87") [86](https://testingbot.com# "FF 86") [85](https://testingbot.com# "FF 85") [84](https://testingbot.com# "FF 84") [83](https://testingbot.com# "FF 83") [82](https://testingbot.com# "FF 82") [81](https://testingbot.com# "FF 81") [80](https://testingbot.com# "FF 80") [79](https://testingbot.com# "FF 79") [78](https://testingbot.com# "FF 78") [77](https://testingbot.com# "FF 77") [76](https://testingbot.com# "FF 76") [75](https://testingbot.com# "FF 75") [74](https://testingbot.com# "FF 74") [73](https://testingbot.com# "FF 73") [72](https://testingbot.com# "FF 72") [71](https://testingbot.com# "FF 71") [70](https://testingbot.com# "FF 70") [69](https://testingbot.com# "FF 69") [68](https://testingbot.com# "FF 68") [67](https://testingbot.com# "FF 67") [66](https://testingbot.com# "FF 66") [65](https://testingbot.com# "FF 65") [64](https://testingbot.com# "FF 64") [63](https://testingbot.com# "FF 63") [62](https://testingbot.com# "FF 62") [61](https://testingbot.com# "FF 61") [60](https://testingbot.com# "FF 60") [59](https://testingbot.com# "FF 59") [58](https://testingbot.com# "FF 58") [57](https://testingbot.com# "FF 57") [56](https://testingbot.com# "FF 56") [55](https://testingbot.com# "FF 55") [54](https://testingbot.com# "FF 54") [53](https://testingbot.com# "FF 53") [52](https://testingbot.com# "FF 52") [51](https://testingbot.com# "FF 51") [50](https://testingbot.com# "FF 50") [49](https://testingbot.com# "FF 49") [48](https://testingbot.com# "FF 48") [47](https://testingbot.com# "FF 47") [46](https://testingbot.com# "FF 46") [45](https://testingbot.com# "FF 45") [44](https://testingbot.com# "FF 44") [43](https://testingbot.com# "FF 43") [42](https://testingbot.com# "FF 42") [41](https://testingbot.com# "FF 41") [40](https://testingbot.com# "FF 40") [39](https://testingbot.com# "FF 39") [38](https://testingbot.com# "FF 38") [37](https://testingbot.com# "FF 37") [36](https://testingbot.com# "FF 36") [35](https://testingbot.com# "FF 35") [34](https://testingbot.com# "FF 34") [33](https://testingbot.com# "FF 33") [32](https://testingbot.com# "FF 32") [31](https://testingbot.com# "FF 31") [30](https://testingbot.com# "FF 30") [29](https://testingbot.com# "FF 29") [28](https://testingbot.com# "FF 28") [27](https://testingbot.com# "FF 27") [26](https://testingbot.com# "FF 26") [25](https://testingbot.com# "FF 25") [24](https://testingbot.com# "FF 24") [23](https://testingbot.com# "FF 23") [22](https://testingbot.com# "FF 22") [21](https://testingbot.com# "FF 21") [20](https://testingbot.com# "FF 20") [19](https://testingbot.com# "FF 19") [18](https://testingbot.com# "FF 18") [17](https://testingbot.com# "FF 17") [16](https://testingbot.com# "FF 16") [15](https://testingbot.com# "FF 15") [14](https://testingbot.com# "FF 14") [13](https://testingbot.com# "FF 13") [12](https://testingbot.com# "FF 12") [11](https://testingbot.com# "FF 11") [10](https://testingbot.com# "FF 10") [9](https://testingbot.com# "FF 9") [8](https://testingbot.com# "FF 8") [7](https://testingbot.com# "FF 7") [6](https://testingbot.com# "FF 6") [5](https://testingbot.com# "FF 5") [4](https://testingbot.com# "FF 4") ![iexplore](https://testingbot.com/assets/environments/svg/ie-075955285f8005808f3d26947aace877f96e5843a33bd4d6e997709d3a294451.svg "Iexplore") ### IE [11](https://testingbot.com# "IE11") ![Windows 8 Browser Testing](https://testingbot.com/assets/environments/svg/windows10-f19b2b5055d9a272b1c8b35711154fc9cca1ae14442b48771e44680d28faffee.svg) ## Windows 8 (64-bit) ![googlechrome](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg "Googlechrome") ### Chrome [109](https://testingbot.com# "Chrome 109") [108](https://testingbot.com# "Chrome 108") [107](https://testingbot.com# "Chrome 107") [106](https://testingbot.com# "Chrome 106") [105](https://testingbot.com# "Chrome 105") [104](https://testingbot.com# "Chrome 104") [103](https://testingbot.com# "Chrome 103") [102](https://testingbot.com# "Chrome 102") [101](https://testingbot.com# "Chrome 101") [100](https://testingbot.com# "Chrome 100") [99](https://testingbot.com# "Chrome 99") [98](https://testingbot.com# "Chrome 98") [97](https://testingbot.com# "Chrome 97") [96](https://testingbot.com# "Chrome 96") [95](https://testingbot.com# "Chrome 95") [94](https://testingbot.com# "Chrome 94") [93](https://testingbot.com# "Chrome 93") [92](https://testingbot.com# "Chrome 92") [91](https://testingbot.com# "Chrome 91") [90](https://testingbot.com# "Chrome 90") [89](https://testingbot.com# "Chrome 89") [88](https://testingbot.com# "Chrome 88") [87](https://testingbot.com# "Chrome 87") [86](https://testingbot.com# "Chrome 86") [85](https://testingbot.com# "Chrome 85") [84](https://testingbot.com# "Chrome 84") [83](https://testingbot.com# "Chrome 83") [81](https://testingbot.com# "Chrome 81") [80](https://testingbot.com# "Chrome 80") [79](https://testingbot.com# "Chrome 79") [78](https://testingbot.com# "Chrome 78") [77](https://testingbot.com# "Chrome 77") [76](https://testingbot.com# "Chrome 76") [75](https://testingbot.com# "Chrome 75") [74](https://testingbot.com# "Chrome 74") [73](https://testingbot.com# "Chrome 73") [72](https://testingbot.com# "Chrome 72") [71](https://testingbot.com# "Chrome 71") [70](https://testingbot.com# "Chrome 70") [69](https://testingbot.com# "Chrome 69") [68](https://testingbot.com# "Chrome 68") [67](https://testingbot.com# "Chrome 67") [66](https://testingbot.com# "Chrome 66") [65](https://testingbot.com# "Chrome 65") [64](https://testingbot.com# "Chrome 64") [63](https://testingbot.com# "Chrome 63") [62](https://testingbot.com# "Chrome 62") [61](https://testingbot.com# "Chrome 61") [60](https://testingbot.com# "Chrome 60") [59](https://testingbot.com# "Chrome 59") [58](https://testingbot.com# "Chrome 58") [57](https://testingbot.com# "Chrome 57") [56](https://testingbot.com# "Chrome 56") [55](https://testingbot.com# "Chrome 55") [54](https://testingbot.com# "Chrome 54") [53](https://testingbot.com# "Chrome 53") [52](https://testingbot.com# "Chrome 52") [51](https://testingbot.com# "Chrome 51") [50](https://testingbot.com# "Chrome 50") [49](https://testingbot.com# "Chrome 49") [48](https://testingbot.com# "Chrome 48") [47](https://testingbot.com# "Chrome 47") [46](https://testingbot.com# "Chrome 46") [45](https://testingbot.com# "Chrome 45") [44](https://testingbot.com# "Chrome 44") [43](https://testingbot.com# "Chrome 43") [42](https://testingbot.com# "Chrome 42") [41](https://testingbot.com# "Chrome 41") [40](https://testingbot.com# "Chrome 40") [39](https://testingbot.com# "Chrome 39") [38](https://testingbot.com# "Chrome 38") [37](https://testingbot.com# "Chrome 37") [36](https://testingbot.com# "Chrome 36") ![firefox](https://testingbot.com/assets/environments/svg/firefox-90dce8de549728e54bd3209cfacb9f1a868c2c8e70d243eb5b930d69cbb2fb1e.svg "Firefox") ### Firefox [115](https://testingbot.com# "FF 115") [114](https://testingbot.com# "FF 114") [113](https://testingbot.com# "FF 113") [112](https://testingbot.com# "FF 112") [111](https://testingbot.com# "FF 111") [110](https://testingbot.com# "FF 110") [109](https://testingbot.com# "FF 109") [108](https://testingbot.com# "FF 108") [107](https://testingbot.com# "FF 107") [106](https://testingbot.com# "FF 106") [105](https://testingbot.com# "FF 105") [104](https://testingbot.com# "FF 104") [103](https://testingbot.com# "FF 103") [102](https://testingbot.com# "FF 102") [101](https://testingbot.com# "FF 101") [100](https://testingbot.com# "FF 100") [99](https://testingbot.com# "FF 99") [98](https://testingbot.com# "FF 98") [97](https://testingbot.com# "FF 97") [96](https://testingbot.com# "FF 96") [95](https://testingbot.com# "FF 95") [94](https://testingbot.com# "FF 94") [93](https://testingbot.com# "FF 93") [92](https://testingbot.com# "FF 92") [91](https://testingbot.com# "FF 91") [90](https://testingbot.com# "FF 90") [89](https://testingbot.com# "FF 89") [88](https://testingbot.com# "FF 88") [87](https://testingbot.com# "FF 87") [86](https://testingbot.com# "FF 86") [85](https://testingbot.com# "FF 85") [84](https://testingbot.com# "FF 84") [83](https://testingbot.com# "FF 83") [82](https://testingbot.com# "FF 82") [81](https://testingbot.com# "FF 81") [80](https://testingbot.com# "FF 80") [79](https://testingbot.com# "FF 79") [78](https://testingbot.com# "FF 78") [77](https://testingbot.com# "FF 77") [76](https://testingbot.com# "FF 76") [75](https://testingbot.com# "FF 75") [74](https://testingbot.com# "FF 74") [73](https://testingbot.com# "FF 73") [72](https://testingbot.com# "FF 72") [71](https://testingbot.com# "FF 71") [70](https://testingbot.com# "FF 70") [69](https://testingbot.com# "FF 69") [68](https://testingbot.com# "FF 68") [67](https://testingbot.com# "FF 67") [66](https://testingbot.com# "FF 66") [65](https://testingbot.com# "FF 65") [64](https://testingbot.com# "FF 64") [63](https://testingbot.com# "FF 63") [62](https://testingbot.com# "FF 62") [61](https://testingbot.com# "FF 61") [60](https://testingbot.com# "FF 60") [59](https://testingbot.com# "FF 59") [58](https://testingbot.com# "FF 58") [57](https://testingbot.com# "FF 57") [56](https://testingbot.com# "FF 56") [55](https://testingbot.com# "FF 55") [54](https://testingbot.com# "FF 54") [53](https://testingbot.com# "FF 53") [52](https://testingbot.com# "FF 52") [51](https://testingbot.com# "FF 51") [50](https://testingbot.com# "FF 50") [49](https://testingbot.com# "FF 49") [48](https://testingbot.com# "FF 48") [47](https://testingbot.com# "FF 47") [46](https://testingbot.com# "FF 46") [45](https://testingbot.com# "FF 45") [44](https://testingbot.com# "FF 44") [43](https://testingbot.com# "FF 43") [42](https://testingbot.com# "FF 42") [41](https://testingbot.com# "FF 41") [40](https://testingbot.com# "FF 40") [39](https://testingbot.com# "FF 39") [38](https://testingbot.com# "FF 38") [37](https://testingbot.com# "FF 37") [36](https://testingbot.com# "FF 36") [35](https://testingbot.com# "FF 35") [34](https://testingbot.com# "FF 34") [33](https://testingbot.com# "FF 33") [32](https://testingbot.com# "FF 32") [31](https://testingbot.com# "FF 31") [30](https://testingbot.com# "FF 30") [29](https://testingbot.com# "FF 29") [28](https://testingbot.com# "FF 28") [27](https://testingbot.com# "FF 27") [26](https://testingbot.com# "FF 26") [25](https://testingbot.com# "FF 25") [24](https://testingbot.com# "FF 24") [23](https://testingbot.com# "FF 23") [22](https://testingbot.com# "FF 22") [21](https://testingbot.com# "FF 21") [20](https://testingbot.com# "FF 20") [19](https://testingbot.com# "FF 19") [18](https://testingbot.com# "FF 18") [17](https://testingbot.com# "FF 17") [16](https://testingbot.com# "FF 16") [15](https://testingbot.com# "FF 15") [14](https://testingbot.com# "FF 14") [13](https://testingbot.com# "FF 13") [12](https://testingbot.com# "FF 12") [11](https://testingbot.com# "FF 11") [10](https://testingbot.com# "FF 10") [9](https://testingbot.com# "FF 9") [8](https://testingbot.com# "FF 8") [7](https://testingbot.com# "FF 7") [6](https://testingbot.com# "FF 6") [5](https://testingbot.com# "FF 5") [4](https://testingbot.com# "FF 4") ![iexplore](https://testingbot.com/assets/environments/svg/ie-075955285f8005808f3d26947aace877f96e5843a33bd4d6e997709d3a294451.svg "Iexplore") ### IE [10](https://testingbot.com# "IE10") ![Windows 7 Browser Testing](https://testingbot.com/assets/environments/svg/windows-18aedb309faf40ff3447063f014404d18f2f97dee5deb28877d5f1e9787b2337.svg) ## Windows 7 (64-bit) ![googlechrome](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg "Googlechrome") ### Chrome [109](https://testingbot.com# "Chrome 109") [108](https://testingbot.com# "Chrome 108") [107](https://testingbot.com# "Chrome 107") [106](https://testingbot.com# "Chrome 106") [105](https://testingbot.com# "Chrome 105") [104](https://testingbot.com# "Chrome 104") [103](https://testingbot.com# "Chrome 103") [102](https://testingbot.com# "Chrome 102") [101](https://testingbot.com# "Chrome 101") [100](https://testingbot.com# "Chrome 100") [99](https://testingbot.com# "Chrome 99") [98](https://testingbot.com# "Chrome 98") [97](https://testingbot.com# "Chrome 97") [96](https://testingbot.com# "Chrome 96") [95](https://testingbot.com# "Chrome 95") [94](https://testingbot.com# "Chrome 94") [93](https://testingbot.com# "Chrome 93") [92](https://testingbot.com# "Chrome 92") [91](https://testingbot.com# "Chrome 91") [90](https://testingbot.com# "Chrome 90") [89](https://testingbot.com# "Chrome 89") [88](https://testingbot.com# "Chrome 88") [87](https://testingbot.com# "Chrome 87") [86](https://testingbot.com# "Chrome 86") [85](https://testingbot.com# "Chrome 85") [84](https://testingbot.com# "Chrome 84") [83](https://testingbot.com# "Chrome 83") [81](https://testingbot.com# "Chrome 81") [80](https://testingbot.com# "Chrome 80") [79](https://testingbot.com# "Chrome 79") [78](https://testingbot.com# "Chrome 78") [77](https://testingbot.com# "Chrome 77") [76](https://testingbot.com# "Chrome 76") [75](https://testingbot.com# "Chrome 75") [74](https://testingbot.com# "Chrome 74") [73](https://testingbot.com# "Chrome 73") [72](https://testingbot.com# "Chrome 72") [71](https://testingbot.com# "Chrome 71") [70](https://testingbot.com# "Chrome 70") [69](https://testingbot.com# "Chrome 69") [68](https://testingbot.com# "Chrome 68") [67](https://testingbot.com# "Chrome 67") [66](https://testingbot.com# "Chrome 66") [65](https://testingbot.com# "Chrome 65") [64](https://testingbot.com# "Chrome 64") [63](https://testingbot.com# "Chrome 63") [62](https://testingbot.com# "Chrome 62") [61](https://testingbot.com# "Chrome 61") [60](https://testingbot.com# "Chrome 60") [59](https://testingbot.com# "Chrome 59") [58](https://testingbot.com# "Chrome 58") [57](https://testingbot.com# "Chrome 57") [56](https://testingbot.com# "Chrome 56") [55](https://testingbot.com# "Chrome 55") [54](https://testingbot.com# "Chrome 54") [53](https://testingbot.com# "Chrome 53") [52](https://testingbot.com# "Chrome 52") [51](https://testingbot.com# "Chrome 51") [50](https://testingbot.com# "Chrome 50") [49](https://testingbot.com# "Chrome 49") [48](https://testingbot.com# "Chrome 48") [47](https://testingbot.com# "Chrome 47") [46](https://testingbot.com# "Chrome 46") [45](https://testingbot.com# "Chrome 45") [44](https://testingbot.com# "Chrome 44") [43](https://testingbot.com# "Chrome 43") [42](https://testingbot.com# "Chrome 42") [41](https://testingbot.com# "Chrome 41") [40](https://testingbot.com# "Chrome 40") [39](https://testingbot.com# "Chrome 39") [38](https://testingbot.com# "Chrome 38") [37](https://testingbot.com# "Chrome 37") [36](https://testingbot.com# "Chrome 36") ![firefox](https://testingbot.com/assets/environments/svg/firefox-90dce8de549728e54bd3209cfacb9f1a868c2c8e70d243eb5b930d69cbb2fb1e.svg "Firefox") ### Firefox [115](https://testingbot.com# "FF 115") [114](https://testingbot.com# "FF 114") [113](https://testingbot.com# "FF 113") [112](https://testingbot.com# "FF 112") [111](https://testingbot.com# "FF 111") [110](https://testingbot.com# "FF 110") [109](https://testingbot.com# "FF 109") [108](https://testingbot.com# "FF 108") [107](https://testingbot.com# "FF 107") [106](https://testingbot.com# "FF 106") [105](https://testingbot.com# "FF 105") [104](https://testingbot.com# "FF 104") [103](https://testingbot.com# "FF 103") [102](https://testingbot.com# "FF 102") [101](https://testingbot.com# "FF 101") [100](https://testingbot.com# "FF 100") [99](https://testingbot.com# "FF 99") [98](https://testingbot.com# "FF 98") [97](https://testingbot.com# "FF 97") [96](https://testingbot.com# "FF 96") [95](https://testingbot.com# "FF 95") [94](https://testingbot.com# "FF 94") [93](https://testingbot.com# "FF 93") [92](https://testingbot.com# "FF 92") [91](https://testingbot.com# "FF 91") [90](https://testingbot.com# "FF 90") [89](https://testingbot.com# "FF 89") [88](https://testingbot.com# "FF 88") [87](https://testingbot.com# "FF 87") [86](https://testingbot.com# "FF 86") [85](https://testingbot.com# "FF 85") [84](https://testingbot.com# "FF 84") [83](https://testingbot.com# "FF 83") [82](https://testingbot.com# "FF 82") [81](https://testingbot.com# "FF 81") [80](https://testingbot.com# "FF 80") [79](https://testingbot.com# "FF 79") [78](https://testingbot.com# "FF 78") [77](https://testingbot.com# "FF 77") [76](https://testingbot.com# "FF 76") [75](https://testingbot.com# "FF 75") [74](https://testingbot.com# "FF 74") [73](https://testingbot.com# "FF 73") [72](https://testingbot.com# "FF 72") [71](https://testingbot.com# "FF 71") [70](https://testingbot.com# "FF 70") [69](https://testingbot.com# "FF 69") [68](https://testingbot.com# "FF 68") [67](https://testingbot.com# "FF 67") [66](https://testingbot.com# "FF 66") [65](https://testingbot.com# "FF 65") [64](https://testingbot.com# "FF 64") [63](https://testingbot.com# "FF 63") [62](https://testingbot.com# "FF 62") [61](https://testingbot.com# "FF 61") [60](https://testingbot.com# "FF 60") [59](https://testingbot.com# "FF 59") [58](https://testingbot.com# "FF 58") [57](https://testingbot.com# "FF 57") [56](https://testingbot.com# "FF 56") [55](https://testingbot.com# "FF 55") [54](https://testingbot.com# "FF 54") [53](https://testingbot.com# "FF 53") [52](https://testingbot.com# "FF 52") [51](https://testingbot.com# "FF 51") [50](https://testingbot.com# "FF 50") [49](https://testingbot.com# "FF 49") [48](https://testingbot.com# "FF 48") [47](https://testingbot.com# "FF 47") [46](https://testingbot.com# "FF 46") [45](https://testingbot.com# "FF 45") [44](https://testingbot.com# "FF 44") [43](https://testingbot.com# "FF 43") [42](https://testingbot.com# "FF 42") [41](https://testingbot.com# "FF 41") [40](https://testingbot.com# "FF 40") [39](https://testingbot.com# "FF 39") [38](https://testingbot.com# "FF 38") [37](https://testingbot.com# "FF 37") [36](https://testingbot.com# "FF 36") [35](https://testingbot.com# "FF 35") [34](https://testingbot.com# "FF 34") [33](https://testingbot.com# "FF 33") [32](https://testingbot.com# "FF 32") [31](https://testingbot.com# "FF 31") [30](https://testingbot.com# "FF 30") [29](https://testingbot.com# "FF 29") [28](https://testingbot.com# "FF 28") [27](https://testingbot.com# "FF 27") [26](https://testingbot.com# "FF 26") [25](https://testingbot.com# "FF 25") [24](https://testingbot.com# "FF 24") [23](https://testingbot.com# "FF 23") [22](https://testingbot.com# "FF 22") [21](https://testingbot.com# "FF 21") [20](https://testingbot.com# "FF 20") [19](https://testingbot.com# "FF 19") [18](https://testingbot.com# "FF 18") [17](https://testingbot.com# "FF 17") [16](https://testingbot.com# "FF 16") [15](https://testingbot.com# "FF 15") [14](https://testingbot.com# "FF 14") [13](https://testingbot.com# "FF 13") [12](https://testingbot.com# "FF 12") [11](https://testingbot.com# "FF 11") [10](https://testingbot.com# "FF 10") [9](https://testingbot.com# "FF 9") [8](https://testingbot.com# "FF 8") [7](https://testingbot.com# "FF 7") [6](https://testingbot.com# "FF 6") [5](https://testingbot.com# "FF 5") [4](https://testingbot.com# "FF 4") ![iexplore](https://testingbot.com/assets/environments/svg/ie-075955285f8005808f3d26947aace877f96e5843a33bd4d6e997709d3a294451.svg "Iexplore") ### IE [11](https://testingbot.com# "IE11") [10](https://testingbot.com# "IE10") [9](https://testingbot.com# "IE9") [8](https://testingbot.com# "IE8") ![opera](https://testingbot.com/assets/environments/svg/opera-fe6e1d716472033125a969dfe6489ab45f0eded91e9260292d6c4ce9b682a330.svg "Opera") ### Opera [95](https://testingbot.com# "Opera 95") [94](https://testingbot.com# "Opera 94") [93](https://testingbot.com# "Opera 93") [92](https://testingbot.com# "Opera 92") [91](https://testingbot.com# "Opera 91") [90](https://testingbot.com# "Opera 90") [89](https://testingbot.com# "Opera 89") [88](https://testingbot.com# "Opera 88") [87](https://testingbot.com# "Opera 87") [86](https://testingbot.com# "Opera 86") [85](https://testingbot.com# "Opera 85") [84](https://testingbot.com# "Opera 84") [83](https://testingbot.com# "Opera 83") [82](https://testingbot.com# "Opera 82") [81](https://testingbot.com# "Opera 81") [80](https://testingbot.com# "Opera 80") [79](https://testingbot.com# "Opera 79") [78](https://testingbot.com# "Opera 78") [77](https://testingbot.com# "Opera 77") [76](https://testingbot.com# "Opera 76") [75](https://testingbot.com# "Opera 75") [74](https://testingbot.com# "Opera 74") [73](https://testingbot.com# "Opera 73") [72](https://testingbot.com# "Opera 72") [71](https://testingbot.com# "Opera 71") [70](https://testingbot.com# "Opera 70") [69](https://testingbot.com# "Opera 69") [68](https://testingbot.com# "Opera 68") [67](https://testingbot.com# "Opera 67") [66](https://testingbot.com# "Opera 66") [65](https://testingbot.com# "Opera 65") [64](https://testingbot.com# "Opera 64") [63](https://testingbot.com# "Opera 63") [62](https://testingbot.com# "Opera 62") [60](https://testingbot.com# "Opera 60") [59](https://testingbot.com# "Opera 59") [58](https://testingbot.com# "Opera 58") [57](https://testingbot.com# "Opera 57") [56](https://testingbot.com# "Opera 56") [55](https://testingbot.com# "Opera 55") [54](https://testingbot.com# "Opera 54") [53](https://testingbot.com# "Opera 53") [52](https://testingbot.com# "Opera 52") [51](https://testingbot.com# "Opera 51") [50](https://testingbot.com# "Opera 50") ![macOS Golden Gate Browser Testing](https://testingbot.com/assets/environments/goldengate_logo-c186d316ab997e287f19e935e295e6e6421e52c309ebcdb54b20adff396714d0.png) ## macOS Golden Gate (27.0) (ARM64) ![googlechrome](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg "Googlechrome") ### Chrome [153](https://testingbot.com# "Chrome 153") [152](https://testingbot.com# "Chrome 152") [151](https://testingbot.com# "Chrome 151") [150](https://testingbot.com# "Chrome 150") [149](https://testingbot.com# "Chrome 149") [148](https://testingbot.com# "Chrome 148") [147](https://testingbot.com# "Chrome 147") [146](https://testingbot.com# "Chrome 146") [145](https://testingbot.com# "Chrome 145") [144](https://testingbot.com# "Chrome 144") [143](https://testingbot.com# "Chrome 143") [142](https://testingbot.com# "Chrome 142") [141](https://testingbot.com# "Chrome 141") [140](https://testingbot.com# "Chrome 140") [139](https://testingbot.com# "Chrome 139") [138](https://testingbot.com# "Chrome 138") [137](https://testingbot.com# "Chrome 137") [136](https://testingbot.com# "Chrome 136") [135](https://testingbot.com# "Chrome 135") [134](https://testingbot.com# "Chrome 134") [133](https://testingbot.com# "Chrome 133") [132](https://testingbot.com# "Chrome 132") [131](https://testingbot.com# "Chrome 131") [130](https://testingbot.com# "Chrome 130") [129](https://testingbot.com# "Chrome 129") [128](https://testingbot.com# "Chrome 128") [127](https://testingbot.com# "Chrome 127") [126](https://testingbot.com# "Chrome 126") [125](https://testingbot.com# "Chrome 125") [124](https://testingbot.com# "Chrome 124") [123](https://testingbot.com# "Chrome 123") [122](https://testingbot.com# "Chrome 122") [121](https://testingbot.com# "Chrome 121") [120](https://testingbot.com# "Chrome 120") [119](https://testingbot.com# "Chrome 119") [118](https://testingbot.com# "Chrome 118") [117](https://testingbot.com# "Chrome 117") [116](https://testingbot.com# "Chrome 116") [115](https://testingbot.com# "Chrome 115") [114](https://testingbot.com# "Chrome 114") [113](https://testingbot.com# "Chrome 113") [112](https://testingbot.com# "Chrome 112") [111](https://testingbot.com# "Chrome 111") [110](https://testingbot.com# "Chrome 110") [109](https://testingbot.com# "Chrome 109") [108](https://testingbot.com# "Chrome 108") [107](https://testingbot.com# "Chrome 107") [106](https://testingbot.com# "Chrome 106") [105](https://testingbot.com# "Chrome 105") [104](https://testingbot.com# "Chrome 104") [103](https://testingbot.com# "Chrome 103") [102](https://testingbot.com# "Chrome 102") [101](https://testingbot.com# "Chrome 101") [100](https://testingbot.com# "Chrome 100") ![firefox](https://testingbot.com/assets/environments/svg/firefox-90dce8de549728e54bd3209cfacb9f1a868c2c8e70d243eb5b930d69cbb2fb1e.svg "Firefox") ### Firefox [dev](https://testingbot.com# "FF dev") [beta](https://testingbot.com# "FF beta") [155](https://testingbot.com# "FF 155") [154](https://testingbot.com# "FF 154") [153](https://testingbot.com# "FF 153") [152](https://testingbot.com# "FF 152") [151](https://testingbot.com# "FF 151") [150](https://testingbot.com# "FF 150") [149](https://testingbot.com# "FF 149") [148](https://testingbot.com# "FF 148") [147](https://testingbot.com# "FF 147") [146](https://testingbot.com# "FF 146") [145](https://testingbot.com# "FF 145") [144](https://testingbot.com# "FF 144") [143](https://testingbot.com# "FF 143") [142](https://testingbot.com# "FF 142") [141](https://testingbot.com# "FF 141") [140](https://testingbot.com# "FF 140") [139](https://testingbot.com# "FF 139") [138](https://testingbot.com# "FF 138") [137](https://testingbot.com# "FF 137") [136](https://testingbot.com# "FF 136") [135](https://testingbot.com# "FF 135") [134](https://testingbot.com# "FF 134") [133](https://testingbot.com# "FF 133") [132](https://testingbot.com# "FF 132") [131](https://testingbot.com# "FF 131") [130](https://testingbot.com# "FF 130") [129](https://testingbot.com# "FF 129") [128](https://testingbot.com# "FF 128") [127](https://testingbot.com# "FF 127") [126](https://testingbot.com# "FF 126") [125](https://testingbot.com# "FF 125") [124](https://testingbot.com# "FF 124") [123](https://testingbot.com# "FF 123") [122](https://testingbot.com# "FF 122") [121](https://testingbot.com# "FF 121") [120](https://testingbot.com# "FF 120") [119](https://testingbot.com# "FF 119") [118](https://testingbot.com# "FF 118") [117](https://testingbot.com# "FF 117") [116](https://testingbot.com# "FF 116") [115](https://testingbot.com# "FF 115") [114](https://testingbot.com# "FF 114") [113](https://testingbot.com# "FF 113") [112](https://testingbot.com# "FF 112") [111](https://testingbot.com# "FF 111") [110](https://testingbot.com# "FF 110") [109](https://testingbot.com# "FF 109") [108](https://testingbot.com# "FF 108") [107](https://testingbot.com# "FF 107") [106](https://testingbot.com# "FF 106") [105](https://testingbot.com# "FF 105") [104](https://testingbot.com# "FF 104") [103](https://testingbot.com# "FF 103") [102](https://testingbot.com# "FF 102") [101](https://testingbot.com# "FF 101") [100](https://testingbot.com# "FF 100") [99](https://testingbot.com# "FF 99") [98](https://testingbot.com# "FF 98") [97](https://testingbot.com# "FF 97") [96](https://testingbot.com# "FF 96") [95](https://testingbot.com# "FF 95") [94](https://testingbot.com# "FF 94") [93](https://testingbot.com# "FF 93") [92](https://testingbot.com# "FF 92") [91](https://testingbot.com# "FF 91") [90](https://testingbot.com# "FF 90") [89](https://testingbot.com# "FF 89") [88](https://testingbot.com# "FF 88") [87](https://testingbot.com# "FF 87") [86](https://testingbot.com# "FF 86") [85](https://testingbot.com# "FF 85") [84](https://testingbot.com# "FF 84") [83](https://testingbot.com# "FF 83") [82](https://testingbot.com# "FF 82") [81](https://testingbot.com# "FF 81") [80](https://testingbot.com# "FF 80") [79](https://testingbot.com# "FF 79") [78](https://testingbot.com# "FF 78") [77](https://testingbot.com# "FF 77") [76](https://testingbot.com# "FF 76") [75](https://testingbot.com# "FF 75") [74](https://testingbot.com# "FF 74") [73](https://testingbot.com# "FF 73") [72](https://testingbot.com# "FF 72") [71](https://testingbot.com# "FF 71") [70](https://testingbot.com# "FF 70") ![microsoftedge](https://testingbot.com/assets/environments/svg/edge-b639cac0c7a843271a0d3a6ccf894cc296064b63b2cd2920209bc5d030f3c70b.svg "Microsoftedge") ### Edge [dev](https://testingbot.com# "Edge") [beta](https://testingbot.com# "Edge") [153](https://testingbot.com# "Edge 153") [152](https://testingbot.com# "Edge 152") [151](https://testingbot.com# "Edge 151") [150](https://testingbot.com# "Edge 150") [149](https://testingbot.com# "Edge 149") [148](https://testingbot.com# "Edge 148") [147](https://testingbot.com# "Edge 147") [146](https://testingbot.com# "Edge 146") [145](https://testingbot.com# "Edge 145") [144](https://testingbot.com# "Edge 144") [143](https://testingbot.com# "Edge 143") [142](https://testingbot.com# "Edge 142") [141](https://testingbot.com# "Edge 141") [140](https://testingbot.com# "Edge 140") [139](https://testingbot.com# "Edge 139") [138](https://testingbot.com# "Edge 138") [137](https://testingbot.com# "Edge 137") [136](https://testingbot.com# "Edge 136") [135](https://testingbot.com# "Edge 135") [134](https://testingbot.com# "Edge 134") [133](https://testingbot.com# "Edge 133") [132](https://testingbot.com# "Edge 132") [131](https://testingbot.com# "Edge 131") [130](https://testingbot.com# "Edge 130") [129](https://testingbot.com# "Edge 129") [128](https://testingbot.com# "Edge 128") [127](https://testingbot.com# "Edge 127") [126](https://testingbot.com# "Edge 126") [125](https://testingbot.com# "Edge 125") [124](https://testingbot.com# "Edge 124") [123](https://testingbot.com# "Edge 123") [122](https://testingbot.com# "Edge 122") [121](https://testingbot.com# "Edge 121") [120](https://testingbot.com# "Edge 120") [119](https://testingbot.com# "Edge 119") [118](https://testingbot.com# "Edge 118") [117](https://testingbot.com# "Edge 117") [116](https://testingbot.com# "Edge 116") [115](https://testingbot.com# "Edge 115") [114](https://testingbot.com# "Edge 114") [113](https://testingbot.com# "Edge 113") [112](https://testingbot.com# "Edge 112") [111](https://testingbot.com# "Edge 111") [110](https://testingbot.com# "Edge 110") [109](https://testingbot.com# "Edge 109") [108](https://testingbot.com# "Edge 108") [107](https://testingbot.com# "Edge 107") [106](https://testingbot.com# "Edge 106") [105](https://testingbot.com# "Edge 105") [104](https://testingbot.com# "Edge 104") [103](https://testingbot.com# "Edge 103") [102](https://testingbot.com# "Edge 102") [101](https://testingbot.com# "Edge 101") [100](https://testingbot.com# "Edge 100") [99](https://testingbot.com# "Edge 99") [98](https://testingbot.com# "Edge 98") [97](https://testingbot.com# "Edge 97") [96](https://testingbot.com# "Edge 96") [95](https://testingbot.com# "Edge 95") [94](https://testingbot.com# "Edge 94") [93](https://testingbot.com# "Edge 93") [92](https://testingbot.com# "Edge 92") [91](https://testingbot.com# "Edge 91") [90](https://testingbot.com# "Edge 90") [89](https://testingbot.com# "Edge 89") [88](https://testingbot.com# "Edge 88") [87](https://testingbot.com# "Edge 87") [86](https://testingbot.com# "Edge 86") [85](https://testingbot.com# "Edge 85") [84](https://testingbot.com# "Edge 84") [83](https://testingbot.com# "Edge 83") [81](https://testingbot.com# "Edge 81") [80](https://testingbot.com# "Edge 80") ![safari](https://testingbot.com/assets/environments/svg/safari-e4837bb5f9a9729fef843af4180372e975c62587808b8ab479f615f3e240e858.svg "Safari") ### Safari [27](https://testingbot.com# "Safari 27") ![macOS Tahoe Browser Testing](https://testingbot.com/assets/environments/tahoe_logo-57c43a6991568db006f6302de4cda5171c2e1d56c4d3c0d81ce39ce7ae908146.png) ## macOS Tahoe (26.0) (ARM64) ![googlechrome](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg "Googlechrome") ### Chrome [153](https://testingbot.com# "Chrome 153") [152](https://testingbot.com# "Chrome 152") [151](https://testingbot.com# "Chrome 151") [150](https://testingbot.com# "Chrome 150") [149](https://testingbot.com# "Chrome 149") [148](https://testingbot.com# "Chrome 148") [147](https://testingbot.com# "Chrome 147") [146](https://testingbot.com# "Chrome 146") [145](https://testingbot.com# "Chrome 145") [144](https://testingbot.com# "Chrome 144") [143](https://testingbot.com# "Chrome 143") [142](https://testingbot.com# "Chrome 142") [141](https://testingbot.com# "Chrome 141") [140](https://testingbot.com# "Chrome 140") [139](https://testingbot.com# "Chrome 139") [138](https://testingbot.com# "Chrome 138") [137](https://testingbot.com# "Chrome 137") [136](https://testingbot.com# "Chrome 136") [135](https://testingbot.com# "Chrome 135") [134](https://testingbot.com# "Chrome 134") [133](https://testingbot.com# "Chrome 133") [132](https://testingbot.com# "Chrome 132") [131](https://testingbot.com# "Chrome 131") [130](https://testingbot.com# "Chrome 130") [129](https://testingbot.com# "Chrome 129") [128](https://testingbot.com# "Chrome 128") [127](https://testingbot.com# "Chrome 127") [126](https://testingbot.com# "Chrome 126") [125](https://testingbot.com# "Chrome 125") [124](https://testingbot.com# "Chrome 124") [123](https://testingbot.com# "Chrome 123") [122](https://testingbot.com# "Chrome 122") [121](https://testingbot.com# "Chrome 121") [120](https://testingbot.com# "Chrome 120") [119](https://testingbot.com# "Chrome 119") [118](https://testingbot.com# "Chrome 118") [117](https://testingbot.com# "Chrome 117") [116](https://testingbot.com# "Chrome 116") [115](https://testingbot.com# "Chrome 115") [114](https://testingbot.com# "Chrome 114") [113](https://testingbot.com# "Chrome 113") [112](https://testingbot.com# "Chrome 112") [111](https://testingbot.com# "Chrome 111") [110](https://testingbot.com# "Chrome 110") [109](https://testingbot.com# "Chrome 109") [108](https://testingbot.com# "Chrome 108") [107](https://testingbot.com# "Chrome 107") [106](https://testingbot.com# "Chrome 106") [105](https://testingbot.com# "Chrome 105") [104](https://testingbot.com# "Chrome 104") [103](https://testingbot.com# "Chrome 103") [102](https://testingbot.com# "Chrome 102") [101](https://testingbot.com# "Chrome 101") [100](https://testingbot.com# "Chrome 100") ![firefox](https://testingbot.com/assets/environments/svg/firefox-90dce8de549728e54bd3209cfacb9f1a868c2c8e70d243eb5b930d69cbb2fb1e.svg "Firefox") ### Firefox [dev](https://testingbot.com# "FF dev") [beta](https://testingbot.com# "FF beta") [155](https://testingbot.com# "FF 155") [154](https://testingbot.com# "FF 154") [153](https://testingbot.com# "FF 153") [152](https://testingbot.com# "FF 152") [151](https://testingbot.com# "FF 151") [150](https://testingbot.com# "FF 150") [149](https://testingbot.com# "FF 149") [148](https://testingbot.com# "FF 148") [147](https://testingbot.com# "FF 147") [146](https://testingbot.com# "FF 146") [145](https://testingbot.com# "FF 145") [144](https://testingbot.com# "FF 144") [143](https://testingbot.com# "FF 143") [142](https://testingbot.com# "FF 142") [141](https://testingbot.com# "FF 141") [140](https://testingbot.com# "FF 140") [139](https://testingbot.com# "FF 139") [138](https://testingbot.com# "FF 138") [137](https://testingbot.com# "FF 137") [136](https://testingbot.com# "FF 136") [135](https://testingbot.com# "FF 135") [134](https://testingbot.com# "FF 134") [133](https://testingbot.com# "FF 133") [132](https://testingbot.com# "FF 132") [131](https://testingbot.com# "FF 131") [130](https://testingbot.com# "FF 130") [129](https://testingbot.com# "FF 129") [128](https://testingbot.com# "FF 128") [127](https://testingbot.com# "FF 127") [126](https://testingbot.com# "FF 126") [125](https://testingbot.com# "FF 125") [124](https://testingbot.com# "FF 124") [123](https://testingbot.com# "FF 123") [122](https://testingbot.com# "FF 122") [121](https://testingbot.com# "FF 121") [120](https://testingbot.com# "FF 120") [119](https://testingbot.com# "FF 119") [118](https://testingbot.com# "FF 118") [117](https://testingbot.com# "FF 117") [116](https://testingbot.com# "FF 116") [115](https://testingbot.com# "FF 115") [114](https://testingbot.com# "FF 114") [113](https://testingbot.com# "FF 113") [112](https://testingbot.com# "FF 112") [111](https://testingbot.com# "FF 111") [110](https://testingbot.com# "FF 110") [109](https://testingbot.com# "FF 109") [108](https://testingbot.com# "FF 108") [107](https://testingbot.com# "FF 107") [106](https://testingbot.com# "FF 106") [105](https://testingbot.com# "FF 105") [104](https://testingbot.com# "FF 104") [103](https://testingbot.com# "FF 103") [102](https://testingbot.com# "FF 102") [101](https://testingbot.com# "FF 101") [100](https://testingbot.com# "FF 100") [99](https://testingbot.com# "FF 99") [98](https://testingbot.com# "FF 98") [97](https://testingbot.com# "FF 97") [96](https://testingbot.com# "FF 96") [95](https://testingbot.com# "FF 95") [94](https://testingbot.com# "FF 94") [93](https://testingbot.com# "FF 93") [92](https://testingbot.com# "FF 92") [91](https://testingbot.com# "FF 91") [90](https://testingbot.com# "FF 90") [89](https://testingbot.com# "FF 89") [88](https://testingbot.com# "FF 88") [87](https://testingbot.com# "FF 87") [86](https://testingbot.com# "FF 86") [85](https://testingbot.com# "FF 85") [84](https://testingbot.com# "FF 84") [83](https://testingbot.com# "FF 83") [82](https://testingbot.com# "FF 82") [81](https://testingbot.com# "FF 81") [80](https://testingbot.com# "FF 80") [79](https://testingbot.com# "FF 79") [78](https://testingbot.com# "FF 78") [77](https://testingbot.com# "FF 77") [76](https://testingbot.com# "FF 76") [75](https://testingbot.com# "FF 75") [74](https://testingbot.com# "FF 74") [73](https://testingbot.com# "FF 73") [72](https://testingbot.com# "FF 72") [71](https://testingbot.com# "FF 71") [70](https://testingbot.com# "FF 70") ![microsoftedge](https://testingbot.com/assets/environments/svg/edge-b639cac0c7a843271a0d3a6ccf894cc296064b63b2cd2920209bc5d030f3c70b.svg "Microsoftedge") ### Edge [dev](https://testingbot.com# "Edge") [beta](https://testingbot.com# "Edge") [153](https://testingbot.com# "Edge 153") [152](https://testingbot.com# "Edge 152") [151](https://testingbot.com# "Edge 151") [150](https://testingbot.com# "Edge 150") [149](https://testingbot.com# "Edge 149") [148](https://testingbot.com# "Edge 148") [147](https://testingbot.com# "Edge 147") [146](https://testingbot.com# "Edge 146") [145](https://testingbot.com# "Edge 145") [144](https://testingbot.com# "Edge 144") [143](https://testingbot.com# "Edge 143") [142](https://testingbot.com# "Edge 142") [141](https://testingbot.com# "Edge 141") [140](https://testingbot.com# "Edge 140") [139](https://testingbot.com# "Edge 139") [138](https://testingbot.com# "Edge 138") [137](https://testingbot.com# "Edge 137") [136](https://testingbot.com# "Edge 136") [135](https://testingbot.com# "Edge 135") [134](https://testingbot.com# "Edge 134") [133](https://testingbot.com# "Edge 133") [132](https://testingbot.com# "Edge 132") [131](https://testingbot.com# "Edge 131") [130](https://testingbot.com# "Edge 130") [129](https://testingbot.com# "Edge 129") [128](https://testingbot.com# "Edge 128") [127](https://testingbot.com# "Edge 127") [126](https://testingbot.com# "Edge 126") [125](https://testingbot.com# "Edge 125") [124](https://testingbot.com# "Edge 124") [123](https://testingbot.com# "Edge 123") [122](https://testingbot.com# "Edge 122") [121](https://testingbot.com# "Edge 121") [120](https://testingbot.com# "Edge 120") [119](https://testingbot.com# "Edge 119") [118](https://testingbot.com# "Edge 118") [117](https://testingbot.com# "Edge 117") [116](https://testingbot.com# "Edge 116") [115](https://testingbot.com# "Edge 115") [114](https://testingbot.com# "Edge 114") [113](https://testingbot.com# "Edge 113") [112](https://testingbot.com# "Edge 112") [111](https://testingbot.com# "Edge 111") [110](https://testingbot.com# "Edge 110") [109](https://testingbot.com# "Edge 109") [108](https://testingbot.com# "Edge 108") [107](https://testingbot.com# "Edge 107") [106](https://testingbot.com# "Edge 106") [105](https://testingbot.com# "Edge 105") [104](https://testingbot.com# "Edge 104") [103](https://testingbot.com# "Edge 103") [102](https://testingbot.com# "Edge 102") [101](https://testingbot.com# "Edge 101") [100](https://testingbot.com# "Edge 100") [99](https://testingbot.com# "Edge 99") [98](https://testingbot.com# "Edge 98") [97](https://testingbot.com# "Edge 97") [96](https://testingbot.com# "Edge 96") [95](https://testingbot.com# "Edge 95") [94](https://testingbot.com# "Edge 94") [93](https://testingbot.com# "Edge 93") [92](https://testingbot.com# "Edge 92") [91](https://testingbot.com# "Edge 91") [90](https://testingbot.com# "Edge 90") [89](https://testingbot.com# "Edge 89") [88](https://testingbot.com# "Edge 88") [87](https://testingbot.com# "Edge 87") [86](https://testingbot.com# "Edge 86") [85](https://testingbot.com# "Edge 85") [84](https://testingbot.com# "Edge 84") [83](https://testingbot.com# "Edge 83") [81](https://testingbot.com# "Edge 81") [80](https://testingbot.com# "Edge 80") ![safari](https://testingbot.com/assets/environments/svg/safari-e4837bb5f9a9729fef843af4180372e975c62587808b8ab479f615f3e240e858.svg "Safari") ### Safari [26](https://testingbot.com# "Safari 26") ![macOS Sequoia Browser Testing](https://testingbot.com/assets/environments/sequoia_logo-44f79649144ee4151cccc4b7e5f154d15d9a7050e81617ce3a7b9be9a88de018.png) ## macOS Sequoia (15.0) (ARM64) ![googlechrome](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg "Googlechrome") ### Chrome [153](https://testingbot.com# "Chrome 153") [152](https://testingbot.com# "Chrome 152") [151](https://testingbot.com# "Chrome 151") [150](https://testingbot.com# "Chrome 150") [149](https://testingbot.com# "Chrome 149") [148](https://testingbot.com# "Chrome 148") [147](https://testingbot.com# "Chrome 147") [146](https://testingbot.com# "Chrome 146") [145](https://testingbot.com# "Chrome 145") [144](https://testingbot.com# "Chrome 144") [143](https://testingbot.com# "Chrome 143") [142](https://testingbot.com# "Chrome 142") [141](https://testingbot.com# "Chrome 141") [140](https://testingbot.com# "Chrome 140") [139](https://testingbot.com# "Chrome 139") [138](https://testingbot.com# "Chrome 138") [137](https://testingbot.com# "Chrome 137") [136](https://testingbot.com# "Chrome 136") [135](https://testingbot.com# "Chrome 135") [134](https://testingbot.com# "Chrome 134") [133](https://testingbot.com# "Chrome 133") [132](https://testingbot.com# "Chrome 132") [131](https://testingbot.com# "Chrome 131") [130](https://testingbot.com# "Chrome 130") [129](https://testingbot.com# "Chrome 129") [128](https://testingbot.com# "Chrome 128") [127](https://testingbot.com# "Chrome 127") [126](https://testingbot.com# "Chrome 126") [125](https://testingbot.com# "Chrome 125") [124](https://testingbot.com# "Chrome 124") [123](https://testingbot.com# "Chrome 123") [122](https://testingbot.com# "Chrome 122") [121](https://testingbot.com# "Chrome 121") [120](https://testingbot.com# "Chrome 120") [119](https://testingbot.com# "Chrome 119") [118](https://testingbot.com# "Chrome 118") [117](https://testingbot.com# "Chrome 117") [116](https://testingbot.com# "Chrome 116") [115](https://testingbot.com# "Chrome 115") [114](https://testingbot.com# "Chrome 114") [113](https://testingbot.com# "Chrome 113") [112](https://testingbot.com# "Chrome 112") [111](https://testingbot.com# "Chrome 111") [110](https://testingbot.com# "Chrome 110") [109](https://testingbot.com# "Chrome 109") [108](https://testingbot.com# "Chrome 108") [107](https://testingbot.com# "Chrome 107") [106](https://testingbot.com# "Chrome 106") [105](https://testingbot.com# "Chrome 105") [104](https://testingbot.com# "Chrome 104") [103](https://testingbot.com# "Chrome 103") [102](https://testingbot.com# "Chrome 102") [101](https://testingbot.com# "Chrome 101") [100](https://testingbot.com# "Chrome 100") ![firefox](https://testingbot.com/assets/environments/svg/firefox-90dce8de549728e54bd3209cfacb9f1a868c2c8e70d243eb5b930d69cbb2fb1e.svg "Firefox") ### Firefox [dev](https://testingbot.com# "FF dev") [beta](https://testingbot.com# "FF beta") [155](https://testingbot.com# "FF 155") [154](https://testingbot.com# "FF 154") [153](https://testingbot.com# "FF 153") [152](https://testingbot.com# "FF 152") [151](https://testingbot.com# "FF 151") [150](https://testingbot.com# "FF 150") [149](https://testingbot.com# "FF 149") [148](https://testingbot.com# "FF 148") [147](https://testingbot.com# "FF 147") [146](https://testingbot.com# "FF 146") [145](https://testingbot.com# "FF 145") [144](https://testingbot.com# "FF 144") [143](https://testingbot.com# "FF 143") [142](https://testingbot.com# "FF 142") [141](https://testingbot.com# "FF 141") [140](https://testingbot.com# "FF 140") [139](https://testingbot.com# "FF 139") [138](https://testingbot.com# "FF 138") [137](https://testingbot.com# "FF 137") [136](https://testingbot.com# "FF 136") [135](https://testingbot.com# "FF 135") [134](https://testingbot.com# "FF 134") [133](https://testingbot.com# "FF 133") [132](https://testingbot.com# "FF 132") [131](https://testingbot.com# "FF 131") [130](https://testingbot.com# "FF 130") [129](https://testingbot.com# "FF 129") [128](https://testingbot.com# "FF 128") [127](https://testingbot.com# "FF 127") [126](https://testingbot.com# "FF 126") [125](https://testingbot.com# "FF 125") [124](https://testingbot.com# "FF 124") [123](https://testingbot.com# "FF 123") [122](https://testingbot.com# "FF 122") [121](https://testingbot.com# "FF 121") [120](https://testingbot.com# "FF 120") [119](https://testingbot.com# "FF 119") [118](https://testingbot.com# "FF 118") [117](https://testingbot.com# "FF 117") [116](https://testingbot.com# "FF 116") [115](https://testingbot.com# "FF 115") [114](https://testingbot.com# "FF 114") [113](https://testingbot.com# "FF 113") [112](https://testingbot.com# "FF 112") [111](https://testingbot.com# "FF 111") [110](https://testingbot.com# "FF 110") [109](https://testingbot.com# "FF 109") [108](https://testingbot.com# "FF 108") [107](https://testingbot.com# "FF 107") [106](https://testingbot.com# "FF 106") [105](https://testingbot.com# "FF 105") [104](https://testingbot.com# "FF 104") [103](https://testingbot.com# "FF 103") [102](https://testingbot.com# "FF 102") [101](https://testingbot.com# "FF 101") [100](https://testingbot.com# "FF 100") [99](https://testingbot.com# "FF 99") [98](https://testingbot.com# "FF 98") [97](https://testingbot.com# "FF 97") [96](https://testingbot.com# "FF 96") [95](https://testingbot.com# "FF 95") [94](https://testingbot.com# "FF 94") [93](https://testingbot.com# "FF 93") [92](https://testingbot.com# "FF 92") [91](https://testingbot.com# "FF 91") [90](https://testingbot.com# "FF 90") [89](https://testingbot.com# "FF 89") [88](https://testingbot.com# "FF 88") [87](https://testingbot.com# "FF 87") [86](https://testingbot.com# "FF 86") [85](https://testingbot.com# "FF 85") [84](https://testingbot.com# "FF 84") [83](https://testingbot.com# "FF 83") [82](https://testingbot.com# "FF 82") [81](https://testingbot.com# "FF 81") [80](https://testingbot.com# "FF 80") [79](https://testingbot.com# "FF 79") [78](https://testingbot.com# "FF 78") [77](https://testingbot.com# "FF 77") [76](https://testingbot.com# "FF 76") [75](https://testingbot.com# "FF 75") [74](https://testingbot.com# "FF 74") [73](https://testingbot.com# "FF 73") [72](https://testingbot.com# "FF 72") [71](https://testingbot.com# "FF 71") [70](https://testingbot.com# "FF 70") ![microsoftedge](https://testingbot.com/assets/environments/svg/edge-b639cac0c7a843271a0d3a6ccf894cc296064b63b2cd2920209bc5d030f3c70b.svg "Microsoftedge") ### Edge [beta](https://testingbot.com# "Edge") [dev](https://testingbot.com# "Edge") [153](https://testingbot.com# "Edge 153") [152](https://testingbot.com# "Edge 152") [151](https://testingbot.com# "Edge 151") [150](https://testingbot.com# "Edge 150") [149](https://testingbot.com# "Edge 149") [148](https://testingbot.com# "Edge 148") [147](https://testingbot.com# "Edge 147") [146](https://testingbot.com# "Edge 146") [145](https://testingbot.com# "Edge 145") [144](https://testingbot.com# "Edge 144") [143](https://testingbot.com# "Edge 143") [142](https://testingbot.com# "Edge 142") [141](https://testingbot.com# "Edge 141") [140](https://testingbot.com# "Edge 140") [139](https://testingbot.com# "Edge 139") [138](https://testingbot.com# "Edge 138") [137](https://testingbot.com# "Edge 137") [136](https://testingbot.com# "Edge 136") [135](https://testingbot.com# "Edge 135") [134](https://testingbot.com# "Edge 134") [133](https://testingbot.com# "Edge 133") [132](https://testingbot.com# "Edge 132") [131](https://testingbot.com# "Edge 131") [130](https://testingbot.com# "Edge 130") [129](https://testingbot.com# "Edge 129") [128](https://testingbot.com# "Edge 128") [127](https://testingbot.com# "Edge 127") [126](https://testingbot.com# "Edge 126") [125](https://testingbot.com# "Edge 125") [124](https://testingbot.com# "Edge 124") [123](https://testingbot.com# "Edge 123") [122](https://testingbot.com# "Edge 122") [121](https://testingbot.com# "Edge 121") [120](https://testingbot.com# "Edge 120") [119](https://testingbot.com# "Edge 119") [118](https://testingbot.com# "Edge 118") [117](https://testingbot.com# "Edge 117") [116](https://testingbot.com# "Edge 116") [115](https://testingbot.com# "Edge 115") [114](https://testingbot.com# "Edge 114") [113](https://testingbot.com# "Edge 113") [112](https://testingbot.com# "Edge 112") [111](https://testingbot.com# "Edge 111") [110](https://testingbot.com# "Edge 110") [109](https://testingbot.com# "Edge 109") [108](https://testingbot.com# "Edge 108") [107](https://testingbot.com# "Edge 107") [106](https://testingbot.com# "Edge 106") [105](https://testingbot.com# "Edge 105") [104](https://testingbot.com# "Edge 104") [103](https://testingbot.com# "Edge 103") [102](https://testingbot.com# "Edge 102") [101](https://testingbot.com# "Edge 101") [100](https://testingbot.com# "Edge 100") [99](https://testingbot.com# "Edge 99") [98](https://testingbot.com# "Edge 98") [97](https://testingbot.com# "Edge 97") [96](https://testingbot.com# "Edge 96") [95](https://testingbot.com# "Edge 95") [94](https://testingbot.com# "Edge 94") [93](https://testingbot.com# "Edge 93") [92](https://testingbot.com# "Edge 92") [91](https://testingbot.com# "Edge 91") [90](https://testingbot.com# "Edge 90") [89](https://testingbot.com# "Edge 89") [88](https://testingbot.com# "Edge 88") [87](https://testingbot.com# "Edge 87") [86](https://testingbot.com# "Edge 86") [85](https://testingbot.com# "Edge 85") [84](https://testingbot.com# "Edge 84") [83](https://testingbot.com# "Edge 83") [81](https://testingbot.com# "Edge 81") [80](https://testingbot.com# "Edge 80") ![safari](https://testingbot.com/assets/environments/svg/safari-e4837bb5f9a9729fef843af4180372e975c62587808b8ab479f615f3e240e858.svg "Safari") ### Safari [dev](https://testingbot.com# "Safari dev") [18](https://testingbot.com# "Safari 18") ![macOS Sonoma Browser Testing](https://testingbot.com/assets/environments/sonoma_logo-aeace52519331cc0c5c6e235d35f7ecb42e73abcbe5377d7518cfae7679cf403.png) ## macOS Sonoma (14.0) (ARM64) ![googlechrome](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg "Googlechrome") ### Chrome [153](https://testingbot.com# "Chrome 153") [152](https://testingbot.com# "Chrome 152") [151](https://testingbot.com# "Chrome 151") [150](https://testingbot.com# "Chrome 150") [149](https://testingbot.com# "Chrome 149") [148](https://testingbot.com# "Chrome 148") [147](https://testingbot.com# "Chrome 147") [146](https://testingbot.com# "Chrome 146") [145](https://testingbot.com# "Chrome 145") [144](https://testingbot.com# "Chrome 144") [143](https://testingbot.com# "Chrome 143") [142](https://testingbot.com# "Chrome 142") [141](https://testingbot.com# "Chrome 141") [140](https://testingbot.com# "Chrome 140") [139](https://testingbot.com# "Chrome 139") [138](https://testingbot.com# "Chrome 138") [137](https://testingbot.com# "Chrome 137") [136](https://testingbot.com# "Chrome 136") [135](https://testingbot.com# "Chrome 135") [134](https://testingbot.com# "Chrome 134") [133](https://testingbot.com# "Chrome 133") [132](https://testingbot.com# "Chrome 132") [131](https://testingbot.com# "Chrome 131") [130](https://testingbot.com# "Chrome 130") [129](https://testingbot.com# "Chrome 129") [128](https://testingbot.com# "Chrome 128") [127](https://testingbot.com# "Chrome 127") [126](https://testingbot.com# "Chrome 126") [125](https://testingbot.com# "Chrome 125") [124](https://testingbot.com# "Chrome 124") [123](https://testingbot.com# "Chrome 123") [122](https://testingbot.com# "Chrome 122") [121](https://testingbot.com# "Chrome 121") [120](https://testingbot.com# "Chrome 120") [119](https://testingbot.com# "Chrome 119") [118](https://testingbot.com# "Chrome 118") [117](https://testingbot.com# "Chrome 117") [116](https://testingbot.com# "Chrome 116") [115](https://testingbot.com# "Chrome 115") [114](https://testingbot.com# "Chrome 114") [113](https://testingbot.com# "Chrome 113") [112](https://testingbot.com# "Chrome 112") [111](https://testingbot.com# "Chrome 111") [110](https://testingbot.com# "Chrome 110") [109](https://testingbot.com# "Chrome 109") [108](https://testingbot.com# "Chrome 108") [107](https://testingbot.com# "Chrome 107") [106](https://testingbot.com# "Chrome 106") [105](https://testingbot.com# "Chrome 105") [104](https://testingbot.com# "Chrome 104") [103](https://testingbot.com# "Chrome 103") [102](https://testingbot.com# "Chrome 102") [101](https://testingbot.com# "Chrome 101") [100](https://testingbot.com# "Chrome 100") ![firefox](https://testingbot.com/assets/environments/svg/firefox-90dce8de549728e54bd3209cfacb9f1a868c2c8e70d243eb5b930d69cbb2fb1e.svg "Firefox") ### Firefox [dev](https://testingbot.com# "FF dev") [beta](https://testingbot.com# "FF beta") [155](https://testingbot.com# "FF 155") [154](https://testingbot.com# "FF 154") [153](https://testingbot.com# "FF 153") [152](https://testingbot.com# "FF 152") [151](https://testingbot.com# "FF 151") [150](https://testingbot.com# "FF 150") [149](https://testingbot.com# "FF 149") [148](https://testingbot.com# "FF 148") [147](https://testingbot.com# "FF 147") [146](https://testingbot.com# "FF 146") [145](https://testingbot.com# "FF 145") [144](https://testingbot.com# "FF 144") [143](https://testingbot.com# "FF 143") [142](https://testingbot.com# "FF 142") [141](https://testingbot.com# "FF 141") [140](https://testingbot.com# "FF 140") [139](https://testingbot.com# "FF 139") [138](https://testingbot.com# "FF 138") [137](https://testingbot.com# "FF 137") [136](https://testingbot.com# "FF 136") [135](https://testingbot.com# "FF 135") [134](https://testingbot.com# "FF 134") [133](https://testingbot.com# "FF 133") [132](https://testingbot.com# "FF 132") [131](https://testingbot.com# "FF 131") [130](https://testingbot.com# "FF 130") [129](https://testingbot.com# "FF 129") [128](https://testingbot.com# "FF 128") [127](https://testingbot.com# "FF 127") [126](https://testingbot.com# "FF 126") [125](https://testingbot.com# "FF 125") [124](https://testingbot.com# "FF 124") [123](https://testingbot.com# "FF 123") [122](https://testingbot.com# "FF 122") [121](https://testingbot.com# "FF 121") [120](https://testingbot.com# "FF 120") [119](https://testingbot.com# "FF 119") [118](https://testingbot.com# "FF 118") [117](https://testingbot.com# "FF 117") [116](https://testingbot.com# "FF 116") [115](https://testingbot.com# "FF 115") [114](https://testingbot.com# "FF 114") [113](https://testingbot.com# "FF 113") [112](https://testingbot.com# "FF 112") [111](https://testingbot.com# "FF 111") [110](https://testingbot.com# "FF 110") [109](https://testingbot.com# "FF 109") [108](https://testingbot.com# "FF 108") [107](https://testingbot.com# "FF 107") [106](https://testingbot.com# "FF 106") [105](https://testingbot.com# "FF 105") [104](https://testingbot.com# "FF 104") [103](https://testingbot.com# "FF 103") [102](https://testingbot.com# "FF 102") [101](https://testingbot.com# "FF 101") [100](https://testingbot.com# "FF 100") [99](https://testingbot.com# "FF 99") [98](https://testingbot.com# "FF 98") [97](https://testingbot.com# "FF 97") [96](https://testingbot.com# "FF 96") [95](https://testingbot.com# "FF 95") [94](https://testingbot.com# "FF 94") [93](https://testingbot.com# "FF 93") [92](https://testingbot.com# "FF 92") [91](https://testingbot.com# "FF 91") [90](https://testingbot.com# "FF 90") [89](https://testingbot.com# "FF 89") [88](https://testingbot.com# "FF 88") [87](https://testingbot.com# "FF 87") [86](https://testingbot.com# "FF 86") [85](https://testingbot.com# "FF 85") [84](https://testingbot.com# "FF 84") [83](https://testingbot.com# "FF 83") [82](https://testingbot.com# "FF 82") [81](https://testingbot.com# "FF 81") [80](https://testingbot.com# "FF 80") [79](https://testingbot.com# "FF 79") [78](https://testingbot.com# "FF 78") [77](https://testingbot.com# "FF 77") [76](https://testingbot.com# "FF 76") [75](https://testingbot.com# "FF 75") [74](https://testingbot.com# "FF 74") [73](https://testingbot.com# "FF 73") [72](https://testingbot.com# "FF 72") [71](https://testingbot.com# "FF 71") [70](https://testingbot.com# "FF 70") ![microsoftedge](https://testingbot.com/assets/environments/svg/edge-b639cac0c7a843271a0d3a6ccf894cc296064b63b2cd2920209bc5d030f3c70b.svg "Microsoftedge") ### Edge [beta](https://testingbot.com# "Edge") [dev](https://testingbot.com# "Edge") [153](https://testingbot.com# "Edge 153") [152](https://testingbot.com# "Edge 152") [151](https://testingbot.com# "Edge 151") [150](https://testingbot.com# "Edge 150") [149](https://testingbot.com# "Edge 149") [148](https://testingbot.com# "Edge 148") [147](https://testingbot.com# "Edge 147") [146](https://testingbot.com# "Edge 146") [145](https://testingbot.com# "Edge 145") [144](https://testingbot.com# "Edge 144") [143](https://testingbot.com# "Edge 143") [142](https://testingbot.com# "Edge 142") [141](https://testingbot.com# "Edge 141") [140](https://testingbot.com# "Edge 140") [139](https://testingbot.com# "Edge 139") [138](https://testingbot.com# "Edge 138") [137](https://testingbot.com# "Edge 137") [136](https://testingbot.com# "Edge 136") [135](https://testingbot.com# "Edge 135") [134](https://testingbot.com# "Edge 134") [133](https://testingbot.com# "Edge 133") [132](https://testingbot.com# "Edge 132") [131](https://testingbot.com# "Edge 131") [130](https://testingbot.com# "Edge 130") [129](https://testingbot.com# "Edge 129") [128](https://testingbot.com# "Edge 128") [127](https://testingbot.com# "Edge 127") [126](https://testingbot.com# "Edge 126") [125](https://testingbot.com# "Edge 125") [124](https://testingbot.com# "Edge 124") [123](https://testingbot.com# "Edge 123") [122](https://testingbot.com# "Edge 122") [121](https://testingbot.com# "Edge 121") [120](https://testingbot.com# "Edge 120") [119](https://testingbot.com# "Edge 119") [118](https://testingbot.com# "Edge 118") [117](https://testingbot.com# "Edge 117") [116](https://testingbot.com# "Edge 116") [115](https://testingbot.com# "Edge 115") [114](https://testingbot.com# "Edge 114") [113](https://testingbot.com# "Edge 113") [112](https://testingbot.com# "Edge 112") [111](https://testingbot.com# "Edge 111") [110](https://testingbot.com# "Edge 110") [109](https://testingbot.com# "Edge 109") [108](https://testingbot.com# "Edge 108") [107](https://testingbot.com# "Edge 107") [106](https://testingbot.com# "Edge 106") [105](https://testingbot.com# "Edge 105") [104](https://testingbot.com# "Edge 104") [103](https://testingbot.com# "Edge 103") [102](https://testingbot.com# "Edge 102") [101](https://testingbot.com# "Edge 101") [100](https://testingbot.com# "Edge 100") [99](https://testingbot.com# "Edge 99") [98](https://testingbot.com# "Edge 98") [97](https://testingbot.com# "Edge 97") [96](https://testingbot.com# "Edge 96") [95](https://testingbot.com# "Edge 95") [94](https://testingbot.com# "Edge 94") [93](https://testingbot.com# "Edge 93") [92](https://testingbot.com# "Edge 92") [91](https://testingbot.com# "Edge 91") [90](https://testingbot.com# "Edge 90") [89](https://testingbot.com# "Edge 89") [88](https://testingbot.com# "Edge 88") [87](https://testingbot.com# "Edge 87") [86](https://testingbot.com# "Edge 86") [85](https://testingbot.com# "Edge 85") [84](https://testingbot.com# "Edge 84") [83](https://testingbot.com# "Edge 83") [81](https://testingbot.com# "Edge 81") [80](https://testingbot.com# "Edge 80") ![safari](https://testingbot.com/assets/environments/svg/safari-e4837bb5f9a9729fef843af4180372e975c62587808b8ab479f615f3e240e858.svg "Safari") ### Safari [beta](https://testingbot.com# "Safari beta") [dev](https://testingbot.com# "Safari dev") [18](https://testingbot.com# "Safari 18") [17](https://testingbot.com# "Safari 17") ![macOS Ventura Browser Testing](https://testingbot.com/assets/environments/ventura_logo-2191cecf16bacb3874bc9b896158b59e375a8d0d79e47665a77188fba804a4ac.png) ## macOS Ventura (13.2) (ARM64) ![googlechrome](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg "Googlechrome") ### Chrome [beta](https://testingbot.com# "Chrome") [153](https://testingbot.com# "Chrome 153") [152](https://testingbot.com# "Chrome 152") [151](https://testingbot.com# "Chrome 151") [150](https://testingbot.com# "Chrome 150") [149](https://testingbot.com# "Chrome 149") [148](https://testingbot.com# "Chrome 148") [147](https://testingbot.com# "Chrome 147") [146](https://testingbot.com# "Chrome 146") [145](https://testingbot.com# "Chrome 145") [144](https://testingbot.com# "Chrome 144") [143](https://testingbot.com# "Chrome 143") [142](https://testingbot.com# "Chrome 142") [141](https://testingbot.com# "Chrome 141") [140](https://testingbot.com# "Chrome 140") [139](https://testingbot.com# "Chrome 139") [138](https://testingbot.com# "Chrome 138") [137](https://testingbot.com# "Chrome 137") [136](https://testingbot.com# "Chrome 136") [135](https://testingbot.com# "Chrome 135") [134](https://testingbot.com# "Chrome 134") [133](https://testingbot.com# "Chrome 133") [132](https://testingbot.com# "Chrome 132") [131](https://testingbot.com# "Chrome 131") [130](https://testingbot.com# "Chrome 130") [129](https://testingbot.com# "Chrome 129") [128](https://testingbot.com# "Chrome 128") [127](https://testingbot.com# "Chrome 127") [126](https://testingbot.com# "Chrome 126") [125](https://testingbot.com# "Chrome 125") [124](https://testingbot.com# "Chrome 124") [123](https://testingbot.com# "Chrome 123") [122](https://testingbot.com# "Chrome 122") [121](https://testingbot.com# "Chrome 121") [120](https://testingbot.com# "Chrome 120") [119](https://testingbot.com# "Chrome 119") [118](https://testingbot.com# "Chrome 118") [117](https://testingbot.com# "Chrome 117") [116](https://testingbot.com# "Chrome 116") [115](https://testingbot.com# "Chrome 115") [114](https://testingbot.com# "Chrome 114") [113](https://testingbot.com# "Chrome 113") [112](https://testingbot.com# "Chrome 112") [111](https://testingbot.com# "Chrome 111") [110](https://testingbot.com# "Chrome 110") [109](https://testingbot.com# "Chrome 109") [108](https://testingbot.com# "Chrome 108") [107](https://testingbot.com# "Chrome 107") [106](https://testingbot.com# "Chrome 106") [105](https://testingbot.com# "Chrome 105") [104](https://testingbot.com# "Chrome 104") [103](https://testingbot.com# "Chrome 103") [102](https://testingbot.com# "Chrome 102") [101](https://testingbot.com# "Chrome 101") [100](https://testingbot.com# "Chrome 100") ![firefox](https://testingbot.com/assets/environments/svg/firefox-90dce8de549728e54bd3209cfacb9f1a868c2c8e70d243eb5b930d69cbb2fb1e.svg "Firefox") ### Firefox [dev](https://testingbot.com# "FF dev") [beta](https://testingbot.com# "FF beta") [155](https://testingbot.com# "FF 155") [154](https://testingbot.com# "FF 154") [153](https://testingbot.com# "FF 153") [152](https://testingbot.com# "FF 152") [151](https://testingbot.com# "FF 151") [150](https://testingbot.com# "FF 150") [149](https://testingbot.com# "FF 149") [148](https://testingbot.com# "FF 148") [147](https://testingbot.com# "FF 147") [146](https://testingbot.com# "FF 146") [145](https://testingbot.com# "FF 145") [144](https://testingbot.com# "FF 144") [143](https://testingbot.com# "FF 143") [142](https://testingbot.com# "FF 142") [141](https://testingbot.com# "FF 141") [140](https://testingbot.com# "FF 140") [139](https://testingbot.com# "FF 139") [138](https://testingbot.com# "FF 138") [137](https://testingbot.com# "FF 137") [136](https://testingbot.com# "FF 136") [135](https://testingbot.com# "FF 135") [134](https://testingbot.com# "FF 134") [133](https://testingbot.com# "FF 133") [132](https://testingbot.com# "FF 132") [131](https://testingbot.com# "FF 131") [130](https://testingbot.com# "FF 130") [129](https://testingbot.com# "FF 129") [128](https://testingbot.com# "FF 128") [127](https://testingbot.com# "FF 127") [126](https://testingbot.com# "FF 126") [125](https://testingbot.com# "FF 125") [124](https://testingbot.com# "FF 124") [123](https://testingbot.com# "FF 123") [122](https://testingbot.com# "FF 122") [121](https://testingbot.com# "FF 121") [120](https://testingbot.com# "FF 120") [119](https://testingbot.com# "FF 119") [118](https://testingbot.com# "FF 118") [117](https://testingbot.com# "FF 117") [116](https://testingbot.com# "FF 116") [115](https://testingbot.com# "FF 115") [114](https://testingbot.com# "FF 114") [113](https://testingbot.com# "FF 113") [112](https://testingbot.com# "FF 112") [111](https://testingbot.com# "FF 111") [110](https://testingbot.com# "FF 110") [109](https://testingbot.com# "FF 109") [108](https://testingbot.com# "FF 108") [107](https://testingbot.com# "FF 107") [106](https://testingbot.com# "FF 106") [105](https://testingbot.com# "FF 105") [104](https://testingbot.com# "FF 104") [103](https://testingbot.com# "FF 103") [102](https://testingbot.com# "FF 102") [101](https://testingbot.com# "FF 101") [100](https://testingbot.com# "FF 100") [99](https://testingbot.com# "FF 99") [98](https://testingbot.com# "FF 98") [97](https://testingbot.com# "FF 97") [96](https://testingbot.com# "FF 96") [95](https://testingbot.com# "FF 95") [94](https://testingbot.com# "FF 94") [93](https://testingbot.com# "FF 93") [92](https://testingbot.com# "FF 92") [91](https://testingbot.com# "FF 91") [90](https://testingbot.com# "FF 90") [89](https://testingbot.com# "FF 89") [88](https://testingbot.com# "FF 88") [87](https://testingbot.com# "FF 87") [86](https://testingbot.com# "FF 86") [85](https://testingbot.com# "FF 85") [84](https://testingbot.com# "FF 84") [83](https://testingbot.com# "FF 83") [82](https://testingbot.com# "FF 82") [81](https://testingbot.com# "FF 81") [80](https://testingbot.com# "FF 80") [79](https://testingbot.com# "FF 79") [78](https://testingbot.com# "FF 78") [77](https://testingbot.com# "FF 77") [76](https://testingbot.com# "FF 76") [75](https://testingbot.com# "FF 75") [74](https://testingbot.com# "FF 74") [73](https://testingbot.com# "FF 73") [72](https://testingbot.com# "FF 72") [71](https://testingbot.com# "FF 71") [70](https://testingbot.com# "FF 70") ![microsoftedge](https://testingbot.com/assets/environments/svg/edge-b639cac0c7a843271a0d3a6ccf894cc296064b63b2cd2920209bc5d030f3c70b.svg "Microsoftedge") ### Edge [dev](https://testingbot.com# "Edge") [beta](https://testingbot.com# "Edge") [153](https://testingbot.com# "Edge 153") [152](https://testingbot.com# "Edge 152") [151](https://testingbot.com# "Edge 151") [150](https://testingbot.com# "Edge 150") [149](https://testingbot.com# "Edge 149") [148](https://testingbot.com# "Edge 148") [147](https://testingbot.com# "Edge 147") [146](https://testingbot.com# "Edge 146") [145](https://testingbot.com# "Edge 145") [144](https://testingbot.com# "Edge 144") [143](https://testingbot.com# "Edge 143") [142](https://testingbot.com# "Edge 142") [141](https://testingbot.com# "Edge 141") [140](https://testingbot.com# "Edge 140") [139](https://testingbot.com# "Edge 139") [138](https://testingbot.com# "Edge 138") [137](https://testingbot.com# "Edge 137") [136](https://testingbot.com# "Edge 136") [135](https://testingbot.com# "Edge 135") [134](https://testingbot.com# "Edge 134") [133](https://testingbot.com# "Edge 133") [132](https://testingbot.com# "Edge 132") [131](https://testingbot.com# "Edge 131") [130](https://testingbot.com# "Edge 130") [129](https://testingbot.com# "Edge 129") [128](https://testingbot.com# "Edge 128") [127](https://testingbot.com# "Edge 127") [126](https://testingbot.com# "Edge 126") [125](https://testingbot.com# "Edge 125") [124](https://testingbot.com# "Edge 124") [123](https://testingbot.com# "Edge 123") [122](https://testingbot.com# "Edge 122") [121](https://testingbot.com# "Edge 121") [120](https://testingbot.com# "Edge 120") [119](https://testingbot.com# "Edge 119") [118](https://testingbot.com# "Edge 118") [117](https://testingbot.com# "Edge 117") [116](https://testingbot.com# "Edge 116") [115](https://testingbot.com# "Edge 115") [114](https://testingbot.com# "Edge 114") [113](https://testingbot.com# "Edge 113") [112](https://testingbot.com# "Edge 112") [111](https://testingbot.com# "Edge 111") [110](https://testingbot.com# "Edge 110") [109](https://testingbot.com# "Edge 109") [108](https://testingbot.com# "Edge 108") [107](https://testingbot.com# "Edge 107") [106](https://testingbot.com# "Edge 106") [105](https://testingbot.com# "Edge 105") [104](https://testingbot.com# "Edge 104") [103](https://testingbot.com# "Edge 103") [102](https://testingbot.com# "Edge 102") [101](https://testingbot.com# "Edge 101") [100](https://testingbot.com# "Edge 100") [99](https://testingbot.com# "Edge 99") [98](https://testingbot.com# "Edge 98") [97](https://testingbot.com# "Edge 97") [96](https://testingbot.com# "Edge 96") [95](https://testingbot.com# "Edge 95") [94](https://testingbot.com# "Edge 94") [93](https://testingbot.com# "Edge 93") [92](https://testingbot.com# "Edge 92") [91](https://testingbot.com# "Edge 91") [90](https://testingbot.com# "Edge 90") [89](https://testingbot.com# "Edge 89") [88](https://testingbot.com# "Edge 88") [87](https://testingbot.com# "Edge 87") [86](https://testingbot.com# "Edge 86") [85](https://testingbot.com# "Edge 85") [84](https://testingbot.com# "Edge 84") [83](https://testingbot.com# "Edge 83") [81](https://testingbot.com# "Edge 81") [80](https://testingbot.com# "Edge 80") ![safari](https://testingbot.com/assets/environments/svg/safari-e4837bb5f9a9729fef843af4180372e975c62587808b8ab479f615f3e240e858.svg "Safari") ### Safari [beta](https://testingbot.com# "Safari beta") [dev](https://testingbot.com# "Safari dev") [16](https://testingbot.com# "Safari 16") ![macOS Monterey Browser Testing](https://testingbot.com/assets/environments/monterey_logo-ea303be67e778d42b57c65e88f1f7d2de9f4c7f9fc3845fc1f9e2000133a144c.png) ## macOS Monterey (12.6) ![googlechrome](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg "Googlechrome") ### Chrome [beta](https://testingbot.com# "Chrome") [dev](https://testingbot.com# "Chrome") [150](https://testingbot.com# "Chrome 150") [149](https://testingbot.com# "Chrome 149") [148](https://testingbot.com# "Chrome 148") [147](https://testingbot.com# "Chrome 147") [146](https://testingbot.com# "Chrome 146") [145](https://testingbot.com# "Chrome 145") [144](https://testingbot.com# "Chrome 144") [143](https://testingbot.com# "Chrome 143") [142](https://testingbot.com# "Chrome 142") [141](https://testingbot.com# "Chrome 141") [140](https://testingbot.com# "Chrome 140") [139](https://testingbot.com# "Chrome 139") [138](https://testingbot.com# "Chrome 138") [137](https://testingbot.com# "Chrome 137") [136](https://testingbot.com# "Chrome 136") [135](https://testingbot.com# "Chrome 135") [134](https://testingbot.com# "Chrome 134") [133](https://testingbot.com# "Chrome 133") [132](https://testingbot.com# "Chrome 132") [131](https://testingbot.com# "Chrome 131") [130](https://testingbot.com# "Chrome 130") [129](https://testingbot.com# "Chrome 129") [128](https://testingbot.com# "Chrome 128") [127](https://testingbot.com# "Chrome 127") [126](https://testingbot.com# "Chrome 126") [125](https://testingbot.com# "Chrome 125") [124](https://testingbot.com# "Chrome 124") [123](https://testingbot.com# "Chrome 123") [122](https://testingbot.com# "Chrome 122") [121](https://testingbot.com# "Chrome 121") [120](https://testingbot.com# "Chrome 120") [119](https://testingbot.com# "Chrome 119") [118](https://testingbot.com# "Chrome 118") [117](https://testingbot.com# "Chrome 117") [116](https://testingbot.com# "Chrome 116") [115](https://testingbot.com# "Chrome 115") [114](https://testingbot.com# "Chrome 114") [113](https://testingbot.com# "Chrome 113") [112](https://testingbot.com# "Chrome 112") [111](https://testingbot.com# "Chrome 111") [110](https://testingbot.com# "Chrome 110") [109](https://testingbot.com# "Chrome 109") [108](https://testingbot.com# "Chrome 108") [107](https://testingbot.com# "Chrome 107") [106](https://testingbot.com# "Chrome 106") [105](https://testingbot.com# "Chrome 105") [104](https://testingbot.com# "Chrome 104") [103](https://testingbot.com# "Chrome 103") [102](https://testingbot.com# "Chrome 102") [101](https://testingbot.com# "Chrome 101") [100](https://testingbot.com# "Chrome 100") [99](https://testingbot.com# "Chrome 99") [98](https://testingbot.com# "Chrome 98") [97](https://testingbot.com# "Chrome 97") [96](https://testingbot.com# "Chrome 96") [95](https://testingbot.com# "Chrome 95") [94](https://testingbot.com# "Chrome 94") [93](https://testingbot.com# "Chrome 93") [92](https://testingbot.com# "Chrome 92") [91](https://testingbot.com# "Chrome 91") [90](https://testingbot.com# "Chrome 90") [89](https://testingbot.com# "Chrome 89") [88](https://testingbot.com# "Chrome 88") [87](https://testingbot.com# "Chrome 87") [86](https://testingbot.com# "Chrome 86") [85](https://testingbot.com# "Chrome 85") [84](https://testingbot.com# "Chrome 84") [83](https://testingbot.com# "Chrome 83") [81](https://testingbot.com# "Chrome 81") [80](https://testingbot.com# "Chrome 80") [79](https://testingbot.com# "Chrome 79") [78](https://testingbot.com# "Chrome 78") [77](https://testingbot.com# "Chrome 77") [76](https://testingbot.com# "Chrome 76") [75](https://testingbot.com# "Chrome 75") [74](https://testingbot.com# "Chrome 74") [73](https://testingbot.com# "Chrome 73") [72](https://testingbot.com# "Chrome 72") [71](https://testingbot.com# "Chrome 71") [70](https://testingbot.com# "Chrome 70") [69](https://testingbot.com# "Chrome 69") [68](https://testingbot.com# "Chrome 68") [67](https://testingbot.com# "Chrome 67") [66](https://testingbot.com# "Chrome 66") [65](https://testingbot.com# "Chrome 65") [64](https://testingbot.com# "Chrome 64") [63](https://testingbot.com# "Chrome 63") [62](https://testingbot.com# "Chrome 62") [61](https://testingbot.com# "Chrome 61") [60](https://testingbot.com# "Chrome 60") [59](https://testingbot.com# "Chrome 59") [58](https://testingbot.com# "Chrome 58") [57](https://testingbot.com# "Chrome 57") [56](https://testingbot.com# "Chrome 56") [55](https://testingbot.com# "Chrome 55") [54](https://testingbot.com# "Chrome 54") [53](https://testingbot.com# "Chrome 53") [52](https://testingbot.com# "Chrome 52") [51](https://testingbot.com# "Chrome 51") [50](https://testingbot.com# "Chrome 50") [49](https://testingbot.com# "Chrome 49") [48](https://testingbot.com# "Chrome 48") [47](https://testingbot.com# "Chrome 47") [46](https://testingbot.com# "Chrome 46") [45](https://testingbot.com# "Chrome 45") [44](https://testingbot.com# "Chrome 44") [43](https://testingbot.com# "Chrome 43") [42](https://testingbot.com# "Chrome 42") [41](https://testingbot.com# "Chrome 41") [40](https://testingbot.com# "Chrome 40") [39](https://testingbot.com# "Chrome 39") [38](https://testingbot.com# "Chrome 38") [36](https://testingbot.com# "Chrome 36") ![firefox](https://testingbot.com/assets/environments/svg/firefox-90dce8de549728e54bd3209cfacb9f1a868c2c8e70d243eb5b930d69cbb2fb1e.svg "Firefox") ### Firefox [dev](https://testingbot.com# "FF dev") [beta](https://testingbot.com# "FF beta") [155](https://testingbot.com# "FF 155") [154](https://testingbot.com# "FF 154") [153](https://testingbot.com# "FF 153") [152](https://testingbot.com# "FF 152") [151](https://testingbot.com# "FF 151") [150](https://testingbot.com# "FF 150") [149](https://testingbot.com# "FF 149") [148](https://testingbot.com# "FF 148") [147](https://testingbot.com# "FF 147") [146](https://testingbot.com# "FF 146") [145](https://testingbot.com# "FF 145") [144](https://testingbot.com# "FF 144") [143](https://testingbot.com# "FF 143") [142](https://testingbot.com# "FF 142") [141](https://testingbot.com# "FF 141") [140](https://testingbot.com# "FF 140") [139](https://testingbot.com# "FF 139") [138](https://testingbot.com# "FF 138") [137](https://testingbot.com# "FF 137") [136](https://testingbot.com# "FF 136") [135](https://testingbot.com# "FF 135") [134](https://testingbot.com# "FF 134") [133](https://testingbot.com# "FF 133") [132](https://testingbot.com# "FF 132") [131](https://testingbot.com# "FF 131") [130](https://testingbot.com# "FF 130") [129](https://testingbot.com# "FF 129") [128](https://testingbot.com# "FF 128") [127](https://testingbot.com# "FF 127") [126](https://testingbot.com# "FF 126") [125](https://testingbot.com# "FF 125") [124](https://testingbot.com# "FF 124") [123](https://testingbot.com# "FF 123") [122](https://testingbot.com# "FF 122") [121](https://testingbot.com# "FF 121") [120](https://testingbot.com# "FF 120") [119](https://testingbot.com# "FF 119") [118](https://testingbot.com# "FF 118") [117](https://testingbot.com# "FF 117") [116](https://testingbot.com# "FF 116") [115](https://testingbot.com# "FF 115") [114](https://testingbot.com# "FF 114") [113](https://testingbot.com# "FF 113") [112](https://testingbot.com# "FF 112") [111](https://testingbot.com# "FF 111") [110](https://testingbot.com# "FF 110") [109](https://testingbot.com# "FF 109") [108](https://testingbot.com# "FF 108") [107](https://testingbot.com# "FF 107") [106](https://testingbot.com# "FF 106") [105](https://testingbot.com# "FF 105") [104](https://testingbot.com# "FF 104") [103](https://testingbot.com# "FF 103") [102](https://testingbot.com# "FF 102") [101](https://testingbot.com# "FF 101") [100](https://testingbot.com# "FF 100") [99](https://testingbot.com# "FF 99") [98](https://testingbot.com# "FF 98") [97](https://testingbot.com# "FF 97") [96](https://testingbot.com# "FF 96") [95](https://testingbot.com# "FF 95") [94](https://testingbot.com# "FF 94") [93](https://testingbot.com# "FF 93") [92](https://testingbot.com# "FF 92") [91](https://testingbot.com# "FF 91") [90](https://testingbot.com# "FF 90") [89](https://testingbot.com# "FF 89") [88](https://testingbot.com# "FF 88") [87](https://testingbot.com# "FF 87") [86](https://testingbot.com# "FF 86") [85](https://testingbot.com# "FF 85") [84](https://testingbot.com# "FF 84") [83](https://testingbot.com# "FF 83") [82](https://testingbot.com# "FF 82") [81](https://testingbot.com# "FF 81") [80](https://testingbot.com# "FF 80") [79](https://testingbot.com# "FF 79") [78](https://testingbot.com# "FF 78") [77](https://testingbot.com# "FF 77") [76](https://testingbot.com# "FF 76") [75](https://testingbot.com# "FF 75") [74](https://testingbot.com# "FF 74") [73](https://testingbot.com# "FF 73") [72](https://testingbot.com# "FF 72") [71](https://testingbot.com# "FF 71") [70](https://testingbot.com# "FF 70") [69](https://testingbot.com# "FF 69") [68](https://testingbot.com# "FF 68") [67](https://testingbot.com# "FF 67") [66](https://testingbot.com# "FF 66") [65](https://testingbot.com# "FF 65") [64](https://testingbot.com# "FF 64") [63](https://testingbot.com# "FF 63") [62](https://testingbot.com# "FF 62") [61](https://testingbot.com# "FF 61") [60](https://testingbot.com# "FF 60") [59](https://testingbot.com# "FF 59") [58](https://testingbot.com# "FF 58") [57](https://testingbot.com# "FF 57") [56](https://testingbot.com# "FF 56") [55](https://testingbot.com# "FF 55") [54](https://testingbot.com# "FF 54") [53](https://testingbot.com# "FF 53") [52](https://testingbot.com# "FF 52") [51](https://testingbot.com# "FF 51") [50](https://testingbot.com# "FF 50") [49](https://testingbot.com# "FF 49") [48](https://testingbot.com# "FF 48") [47](https://testingbot.com# "FF 47") [46](https://testingbot.com# "FF 46") [45](https://testingbot.com# "FF 45") [44](https://testingbot.com# "FF 44") [43](https://testingbot.com# "FF 43") [42](https://testingbot.com# "FF 42") [41](https://testingbot.com# "FF 41") [40](https://testingbot.com# "FF 40") [39](https://testingbot.com# "FF 39") [38](https://testingbot.com# "FF 38") [37](https://testingbot.com# "FF 37") [36](https://testingbot.com# "FF 36") [35](https://testingbot.com# "FF 35") [34](https://testingbot.com# "FF 34") [33](https://testingbot.com# "FF 33") [32](https://testingbot.com# "FF 32") [31](https://testingbot.com# "FF 31") [30](https://testingbot.com# "FF 30") [29](https://testingbot.com# "FF 29") [28](https://testingbot.com# "FF 28") [27](https://testingbot.com# "FF 27") [26](https://testingbot.com# "FF 26") [25](https://testingbot.com# "FF 25") [24](https://testingbot.com# "FF 24") [23](https://testingbot.com# "FF 23") [22](https://testingbot.com# "FF 22") [21](https://testingbot.com# "FF 21") [20](https://testingbot.com# "FF 20") [19](https://testingbot.com# "FF 19") [18](https://testingbot.com# "FF 18") [17](https://testingbot.com# "FF 17") [16](https://testingbot.com# "FF 16") [15](https://testingbot.com# "FF 15") [14](https://testingbot.com# "FF 14") [13](https://testingbot.com# "FF 13") [12](https://testingbot.com# "FF 12") [11](https://testingbot.com# "FF 11") [10](https://testingbot.com# "FF 10") [9](https://testingbot.com# "FF 9") [8](https://testingbot.com# "FF 8") [7](https://testingbot.com# "FF 7") [6](https://testingbot.com# "FF 6") [5](https://testingbot.com# "FF 5") [4](https://testingbot.com# "FF 4") ![microsoftedge](https://testingbot.com/assets/environments/svg/edge-b639cac0c7a843271a0d3a6ccf894cc296064b63b2cd2920209bc5d030f3c70b.svg "Microsoftedge") ### Edge [dev](https://testingbot.com# "Edge") [150](https://testingbot.com# "Edge 150") [149](https://testingbot.com# "Edge 149") [148](https://testingbot.com# "Edge 148") [147](https://testingbot.com# "Edge 147") [146](https://testingbot.com# "Edge 146") [145](https://testingbot.com# "Edge 145") [144](https://testingbot.com# "Edge 144") [143](https://testingbot.com# "Edge 143") [142](https://testingbot.com# "Edge 142") [141](https://testingbot.com# "Edge 141") [140](https://testingbot.com# "Edge 140") [139](https://testingbot.com# "Edge 139") [138](https://testingbot.com# "Edge 138") [137](https://testingbot.com# "Edge 137") [136](https://testingbot.com# "Edge 136") [135](https://testingbot.com# "Edge 135") [134](https://testingbot.com# "Edge 134") [133](https://testingbot.com# "Edge 133") [132](https://testingbot.com# "Edge 132") [131](https://testingbot.com# "Edge 131") [130](https://testingbot.com# "Edge 130") [129](https://testingbot.com# "Edge 129") [128](https://testingbot.com# "Edge 128") [127](https://testingbot.com# "Edge 127") [126](https://testingbot.com# "Edge 126") [125](https://testingbot.com# "Edge 125") [124](https://testingbot.com# "Edge 124") [123](https://testingbot.com# "Edge 123") [122](https://testingbot.com# "Edge 122") [121](https://testingbot.com# "Edge 121") [120](https://testingbot.com# "Edge 120") [119](https://testingbot.com# "Edge 119") [118](https://testingbot.com# "Edge 118") [117](https://testingbot.com# "Edge 117") [116](https://testingbot.com# "Edge 116") [115](https://testingbot.com# "Edge 115") [114](https://testingbot.com# "Edge 114") [113](https://testingbot.com# "Edge 113") [112](https://testingbot.com# "Edge 112") [111](https://testingbot.com# "Edge 111") [110](https://testingbot.com# "Edge 110") [109](https://testingbot.com# "Edge 109") [108](https://testingbot.com# "Edge 108") [107](https://testingbot.com# "Edge 107") [106](https://testingbot.com# "Edge 106") [105](https://testingbot.com# "Edge 105") [104](https://testingbot.com# "Edge 104") [103](https://testingbot.com# "Edge 103") [102](https://testingbot.com# "Edge 102") [101](https://testingbot.com# "Edge 101") [100](https://testingbot.com# "Edge 100") [99](https://testingbot.com# "Edge 99") [98](https://testingbot.com# "Edge 98") [97](https://testingbot.com# "Edge 97") [96](https://testingbot.com# "Edge 96") [95](https://testingbot.com# "Edge 95") [94](https://testingbot.com# "Edge 94") [93](https://testingbot.com# "Edge 93") [92](https://testingbot.com# "Edge 92") [91](https://testingbot.com# "Edge 91") [90](https://testingbot.com# "Edge 90") [89](https://testingbot.com# "Edge 89") [88](https://testingbot.com# "Edge 88") [87](https://testingbot.com# "Edge 87") [86](https://testingbot.com# "Edge 86") [85](https://testingbot.com# "Edge 85") [84](https://testingbot.com# "Edge 84") [83](https://testingbot.com# "Edge 83") [81](https://testingbot.com# "Edge 81") [80](https://testingbot.com# "Edge 80") ![opera](https://testingbot.com/assets/environments/svg/opera-fe6e1d716472033125a969dfe6489ab45f0eded91e9260292d6c4ce9b682a330.svg "Opera") ### Opera [135](https://testingbot.com# "Opera 135") [134](https://testingbot.com# "Opera 134") [130](https://testingbot.com# "Opera 130") [129](https://testingbot.com# "Opera 129") [128](https://testingbot.com# "Opera 128") [127](https://testingbot.com# "Opera 127") [126](https://testingbot.com# "Opera 126") [125](https://testingbot.com# "Opera 125") [124](https://testingbot.com# "Opera 124") [123](https://testingbot.com# "Opera 123") [122](https://testingbot.com# "Opera 122") [121](https://testingbot.com# "Opera 121") [120](https://testingbot.com# "Opera 120") [119](https://testingbot.com# "Opera 119") [118](https://testingbot.com# "Opera 118") [117](https://testingbot.com# "Opera 117") [116](https://testingbot.com# "Opera 116") [115](https://testingbot.com# "Opera 115") [114](https://testingbot.com# "Opera 114") [113](https://testingbot.com# "Opera 113") [112](https://testingbot.com# "Opera 112") [111](https://testingbot.com# "Opera 111") [110](https://testingbot.com# "Opera 110") [109](https://testingbot.com# "Opera 109") [108](https://testingbot.com# "Opera 108") [107](https://testingbot.com# "Opera 107") [106](https://testingbot.com# "Opera 106") [105](https://testingbot.com# "Opera 105") [104](https://testingbot.com# "Opera 104") [103](https://testingbot.com# "Opera 103") [102](https://testingbot.com# "Opera 102") [101](https://testingbot.com# "Opera 101") [100](https://testingbot.com# "Opera 100") [99](https://testingbot.com# "Opera 99") [98](https://testingbot.com# "Opera 98") [97](https://testingbot.com# "Opera 97") [96](https://testingbot.com# "Opera 96") [95](https://testingbot.com# "Opera 95") [94](https://testingbot.com# "Opera 94") [93](https://testingbot.com# "Opera 93") [92](https://testingbot.com# "Opera 92") [91](https://testingbot.com# "Opera 91") [90](https://testingbot.com# "Opera 90") [89](https://testingbot.com# "Opera 89") [88](https://testingbot.com# "Opera 88") [87](https://testingbot.com# "Opera 87") [86](https://testingbot.com# "Opera 86") [85](https://testingbot.com# "Opera 85") [84](https://testingbot.com# "Opera 84") [83](https://testingbot.com# "Opera 83") [82](https://testingbot.com# "Opera 82") [81](https://testingbot.com# "Opera 81") [80](https://testingbot.com# "Opera 80") [79](https://testingbot.com# "Opera 79") [78](https://testingbot.com# "Opera 78") [77](https://testingbot.com# "Opera 77") [76](https://testingbot.com# "Opera 76") [75](https://testingbot.com# "Opera 75") [74](https://testingbot.com# "Opera 74") [73](https://testingbot.com# "Opera 73") [72](https://testingbot.com# "Opera 72") [71](https://testingbot.com# "Opera 71") [70](https://testingbot.com# "Opera 70") [69](https://testingbot.com# "Opera 69") [68](https://testingbot.com# "Opera 68") [67](https://testingbot.com# "Opera 67") [66](https://testingbot.com# "Opera 66") [65](https://testingbot.com# "Opera 65") [64](https://testingbot.com# "Opera 64") [63](https://testingbot.com# "Opera 63") [62](https://testingbot.com# "Opera 62") [60](https://testingbot.com# "Opera 60") [59](https://testingbot.com# "Opera 59") [58](https://testingbot.com# "Opera 58") [57](https://testingbot.com# "Opera 57") [56](https://testingbot.com# "Opera 56") [55](https://testingbot.com# "Opera 55") [54](https://testingbot.com# "Opera 54") [53](https://testingbot.com# "Opera 53") [52](https://testingbot.com# "Opera 52") [51](https://testingbot.com# "Opera 51") [50](https://testingbot.com# "Opera 50") ![safari](https://testingbot.com/assets/environments/svg/safari-e4837bb5f9a9729fef843af4180372e975c62587808b8ab479f615f3e240e858.svg "Safari") ### Safari [beta](https://testingbot.com# "Safari beta") [dev](https://testingbot.com# "Safari dev") [15](https://testingbot.com# "Safari 15") ![macOS Big Sur Browser Testing](https://testingbot.com/assets/environments/bigsur_logo-a12e9501ec8d615b2196728977e14aabf6e10961fc5fcfb51670f949e50f8364.png) ## macOS Big Sur (11.6) ![googlechrome](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg "Googlechrome") ### Chrome [beta](https://testingbot.com# "Chrome") [dev](https://testingbot.com# "Chrome") [138](https://testingbot.com# "Chrome 138") [137](https://testingbot.com# "Chrome 137") [136](https://testingbot.com# "Chrome 136") [135](https://testingbot.com# "Chrome 135") [134](https://testingbot.com# "Chrome 134") [133](https://testingbot.com# "Chrome 133") [132](https://testingbot.com# "Chrome 132") [131](https://testingbot.com# "Chrome 131") [130](https://testingbot.com# "Chrome 130") [129](https://testingbot.com# "Chrome 129") [128](https://testingbot.com# "Chrome 128") [127](https://testingbot.com# "Chrome 127") [126](https://testingbot.com# "Chrome 126") [125](https://testingbot.com# "Chrome 125") [124](https://testingbot.com# "Chrome 124") [123](https://testingbot.com# "Chrome 123") [122](https://testingbot.com# "Chrome 122") [121](https://testingbot.com# "Chrome 121") [120](https://testingbot.com# "Chrome 120") [119](https://testingbot.com# "Chrome 119") [118](https://testingbot.com# "Chrome 118") [117](https://testingbot.com# "Chrome 117") [116](https://testingbot.com# "Chrome 116") [115](https://testingbot.com# "Chrome 115") [114](https://testingbot.com# "Chrome 114") [113](https://testingbot.com# "Chrome 113") [112](https://testingbot.com# "Chrome 112") [111](https://testingbot.com# "Chrome 111") [110](https://testingbot.com# "Chrome 110") [109](https://testingbot.com# "Chrome 109") [108](https://testingbot.com# "Chrome 108") [107](https://testingbot.com# "Chrome 107") [106](https://testingbot.com# "Chrome 106") [105](https://testingbot.com# "Chrome 105") [104](https://testingbot.com# "Chrome 104") [103](https://testingbot.com# "Chrome 103") [102](https://testingbot.com# "Chrome 102") [101](https://testingbot.com# "Chrome 101") [100](https://testingbot.com# "Chrome 100") [99](https://testingbot.com# "Chrome 99") [98](https://testingbot.com# "Chrome 98") [97](https://testingbot.com# "Chrome 97") [96](https://testingbot.com# "Chrome 96") [95](https://testingbot.com# "Chrome 95") [94](https://testingbot.com# "Chrome 94") [93](https://testingbot.com# "Chrome 93") [92](https://testingbot.com# "Chrome 92") [91](https://testingbot.com# "Chrome 91") [90](https://testingbot.com# "Chrome 90") [89](https://testingbot.com# "Chrome 89") [88](https://testingbot.com# "Chrome 88") [87](https://testingbot.com# "Chrome 87") [86](https://testingbot.com# "Chrome 86") [85](https://testingbot.com# "Chrome 85") [84](https://testingbot.com# "Chrome 84") [83](https://testingbot.com# "Chrome 83") [81](https://testingbot.com# "Chrome 81") [80](https://testingbot.com# "Chrome 80") [79](https://testingbot.com# "Chrome 79") [78](https://testingbot.com# "Chrome 78") [77](https://testingbot.com# "Chrome 77") [76](https://testingbot.com# "Chrome 76") [75](https://testingbot.com# "Chrome 75") [74](https://testingbot.com# "Chrome 74") [73](https://testingbot.com# "Chrome 73") [72](https://testingbot.com# "Chrome 72") [71](https://testingbot.com# "Chrome 71") [70](https://testingbot.com# "Chrome 70") [69](https://testingbot.com# "Chrome 69") [68](https://testingbot.com# "Chrome 68") [67](https://testingbot.com# "Chrome 67") [66](https://testingbot.com# "Chrome 66") [65](https://testingbot.com# "Chrome 65") [64](https://testingbot.com# "Chrome 64") [63](https://testingbot.com# "Chrome 63") [62](https://testingbot.com# "Chrome 62") [61](https://testingbot.com# "Chrome 61") [60](https://testingbot.com# "Chrome 60") [59](https://testingbot.com# "Chrome 59") [58](https://testingbot.com# "Chrome 58") [57](https://testingbot.com# "Chrome 57") [56](https://testingbot.com# "Chrome 56") [55](https://testingbot.com# "Chrome 55") [54](https://testingbot.com# "Chrome 54") [53](https://testingbot.com# "Chrome 53") [52](https://testingbot.com# "Chrome 52") [51](https://testingbot.com# "Chrome 51") [50](https://testingbot.com# "Chrome 50") [49](https://testingbot.com# "Chrome 49") [48](https://testingbot.com# "Chrome 48") [47](https://testingbot.com# "Chrome 47") [46](https://testingbot.com# "Chrome 46") [45](https://testingbot.com# "Chrome 45") [44](https://testingbot.com# "Chrome 44") [43](https://testingbot.com# "Chrome 43") [42](https://testingbot.com# "Chrome 42") [41](https://testingbot.com# "Chrome 41") [40](https://testingbot.com# "Chrome 40") [39](https://testingbot.com# "Chrome 39") [38](https://testingbot.com# "Chrome 38") [36](https://testingbot.com# "Chrome 36") ![firefox](https://testingbot.com/assets/environments/svg/firefox-90dce8de549728e54bd3209cfacb9f1a868c2c8e70d243eb5b930d69cbb2fb1e.svg "Firefox") ### Firefox [dev](https://testingbot.com# "FF dev") [beta](https://testingbot.com# "FF beta") [155](https://testingbot.com# "FF 155") [154](https://testingbot.com# "FF 154") [153](https://testingbot.com# "FF 153") [152](https://testingbot.com# "FF 152") [151](https://testingbot.com# "FF 151") [150](https://testingbot.com# "FF 150") [149](https://testingbot.com# "FF 149") [148](https://testingbot.com# "FF 148") [147](https://testingbot.com# "FF 147") [146](https://testingbot.com# "FF 146") [145](https://testingbot.com# "FF 145") [144](https://testingbot.com# "FF 144") [143](https://testingbot.com# "FF 143") [142](https://testingbot.com# "FF 142") [141](https://testingbot.com# "FF 141") [140](https://testingbot.com# "FF 140") [139](https://testingbot.com# "FF 139") [138](https://testingbot.com# "FF 138") [137](https://testingbot.com# "FF 137") [136](https://testingbot.com# "FF 136") [135](https://testingbot.com# "FF 135") [134](https://testingbot.com# "FF 134") [133](https://testingbot.com# "FF 133") [132](https://testingbot.com# "FF 132") [131](https://testingbot.com# "FF 131") [130](https://testingbot.com# "FF 130") [129](https://testingbot.com# "FF 129") [128](https://testingbot.com# "FF 128") [127](https://testingbot.com# "FF 127") [126](https://testingbot.com# "FF 126") [125](https://testingbot.com# "FF 125") [124](https://testingbot.com# "FF 124") [123](https://testingbot.com# "FF 123") [122](https://testingbot.com# "FF 122") [121](https://testingbot.com# "FF 121") [120](https://testingbot.com# "FF 120") [119](https://testingbot.com# "FF 119") [118](https://testingbot.com# "FF 118") [117](https://testingbot.com# "FF 117") [116](https://testingbot.com# "FF 116") [115](https://testingbot.com# "FF 115") [114](https://testingbot.com# "FF 114") [113](https://testingbot.com# "FF 113") [112](https://testingbot.com# "FF 112") [111](https://testingbot.com# "FF 111") [110](https://testingbot.com# "FF 110") [109](https://testingbot.com# "FF 109") [108](https://testingbot.com# "FF 108") [107](https://testingbot.com# "FF 107") [106](https://testingbot.com# "FF 106") [105](https://testingbot.com# "FF 105") [104](https://testingbot.com# "FF 104") [103](https://testingbot.com# "FF 103") [102](https://testingbot.com# "FF 102") [101](https://testingbot.com# "FF 101") [100](https://testingbot.com# "FF 100") [99](https://testingbot.com# "FF 99") [98](https://testingbot.com# "FF 98") [97](https://testingbot.com# "FF 97") [96](https://testingbot.com# "FF 96") [95](https://testingbot.com# "FF 95") [94](https://testingbot.com# "FF 94") [93](https://testingbot.com# "FF 93") [92](https://testingbot.com# "FF 92") [91](https://testingbot.com# "FF 91") [90](https://testingbot.com# "FF 90") [89](https://testingbot.com# "FF 89") [88](https://testingbot.com# "FF 88") [87](https://testingbot.com# "FF 87") [86](https://testingbot.com# "FF 86") [85](https://testingbot.com# "FF 85") [84](https://testingbot.com# "FF 84") [83](https://testingbot.com# "FF 83") [82](https://testingbot.com# "FF 82") [81](https://testingbot.com# "FF 81") [80](https://testingbot.com# "FF 80") [79](https://testingbot.com# "FF 79") [78](https://testingbot.com# "FF 78") [77](https://testingbot.com# "FF 77") [76](https://testingbot.com# "FF 76") [75](https://testingbot.com# "FF 75") [74](https://testingbot.com# "FF 74") [73](https://testingbot.com# "FF 73") [72](https://testingbot.com# "FF 72") [71](https://testingbot.com# "FF 71") [70](https://testingbot.com# "FF 70") [69](https://testingbot.com# "FF 69") [68](https://testingbot.com# "FF 68") [67](https://testingbot.com# "FF 67") [66](https://testingbot.com# "FF 66") [65](https://testingbot.com# "FF 65") [64](https://testingbot.com# "FF 64") [63](https://testingbot.com# "FF 63") [62](https://testingbot.com# "FF 62") [61](https://testingbot.com# "FF 61") [60](https://testingbot.com# "FF 60") [59](https://testingbot.com# "FF 59") [58](https://testingbot.com# "FF 58") [57](https://testingbot.com# "FF 57") [56](https://testingbot.com# "FF 56") [55](https://testingbot.com# "FF 55") [54](https://testingbot.com# "FF 54") [53](https://testingbot.com# "FF 53") [52](https://testingbot.com# "FF 52") [51](https://testingbot.com# "FF 51") [50](https://testingbot.com# "FF 50") [49](https://testingbot.com# "FF 49") [48](https://testingbot.com# "FF 48") [47](https://testingbot.com# "FF 47") [46](https://testingbot.com# "FF 46") [45](https://testingbot.com# "FF 45") [44](https://testingbot.com# "FF 44") [43](https://testingbot.com# "FF 43") [42](https://testingbot.com# "FF 42") [41](https://testingbot.com# "FF 41") [40](https://testingbot.com# "FF 40") [39](https://testingbot.com# "FF 39") [38](https://testingbot.com# "FF 38") [37](https://testingbot.com# "FF 37") [36](https://testingbot.com# "FF 36") [35](https://testingbot.com# "FF 35") [34](https://testingbot.com# "FF 34") [33](https://testingbot.com# "FF 33") [32](https://testingbot.com# "FF 32") [31](https://testingbot.com# "FF 31") [30](https://testingbot.com# "FF 30") [29](https://testingbot.com# "FF 29") [28](https://testingbot.com# "FF 28") [27](https://testingbot.com# "FF 27") [26](https://testingbot.com# "FF 26") [25](https://testingbot.com# "FF 25") [24](https://testingbot.com# "FF 24") [23](https://testingbot.com# "FF 23") [22](https://testingbot.com# "FF 22") [21](https://testingbot.com# "FF 21") [20](https://testingbot.com# "FF 20") [19](https://testingbot.com# "FF 19") [18](https://testingbot.com# "FF 18") [17](https://testingbot.com# "FF 17") [16](https://testingbot.com# "FF 16") [15](https://testingbot.com# "FF 15") [14](https://testingbot.com# "FF 14") [13](https://testingbot.com# "FF 13") [12](https://testingbot.com# "FF 12") [11](https://testingbot.com# "FF 11") [10](https://testingbot.com# "FF 10") [9](https://testingbot.com# "FF 9") [8](https://testingbot.com# "FF 8") [7](https://testingbot.com# "FF 7") [6](https://testingbot.com# "FF 6") [5](https://testingbot.com# "FF 5") [4](https://testingbot.com# "FF 4") ![microsoftedge](https://testingbot.com/assets/environments/svg/edge-b639cac0c7a843271a0d3a6ccf894cc296064b63b2cd2920209bc5d030f3c70b.svg "Microsoftedge") ### Edge [138](https://testingbot.com# "Edge 138") [137](https://testingbot.com# "Edge 137") [136](https://testingbot.com# "Edge 136") [135](https://testingbot.com# "Edge 135") [134](https://testingbot.com# "Edge 134") [133](https://testingbot.com# "Edge 133") [132](https://testingbot.com# "Edge 132") [131](https://testingbot.com# "Edge 131") [130](https://testingbot.com# "Edge 130") [129](https://testingbot.com# "Edge 129") [128](https://testingbot.com# "Edge 128") [127](https://testingbot.com# "Edge 127") [126](https://testingbot.com# "Edge 126") [125](https://testingbot.com# "Edge 125") [124](https://testingbot.com# "Edge 124") [123](https://testingbot.com# "Edge 123") [122](https://testingbot.com# "Edge 122") [121](https://testingbot.com# "Edge 121") [120](https://testingbot.com# "Edge 120") [119](https://testingbot.com# "Edge 119") [118](https://testingbot.com# "Edge 118") [117](https://testingbot.com# "Edge 117") [116](https://testingbot.com# "Edge 116") [115](https://testingbot.com# "Edge 115") [114](https://testingbot.com# "Edge 114") [113](https://testingbot.com# "Edge 113") [112](https://testingbot.com# "Edge 112") [111](https://testingbot.com# "Edge 111") [110](https://testingbot.com# "Edge 110") [109](https://testingbot.com# "Edge 109") [108](https://testingbot.com# "Edge 108") [107](https://testingbot.com# "Edge 107") [106](https://testingbot.com# "Edge 106") [105](https://testingbot.com# "Edge 105") [104](https://testingbot.com# "Edge 104") [103](https://testingbot.com# "Edge 103") [102](https://testingbot.com# "Edge 102") [101](https://testingbot.com# "Edge 101") [100](https://testingbot.com# "Edge 100") [99](https://testingbot.com# "Edge 99") [98](https://testingbot.com# "Edge 98") [97](https://testingbot.com# "Edge 97") [96](https://testingbot.com# "Edge 96") [95](https://testingbot.com# "Edge 95") [94](https://testingbot.com# "Edge 94") [93](https://testingbot.com# "Edge 93") [92](https://testingbot.com# "Edge 92") [91](https://testingbot.com# "Edge 91") [90](https://testingbot.com# "Edge 90") [89](https://testingbot.com# "Edge 89") [88](https://testingbot.com# "Edge 88") [87](https://testingbot.com# "Edge 87") [86](https://testingbot.com# "Edge 86") [85](https://testingbot.com# "Edge 85") [84](https://testingbot.com# "Edge 84") [83](https://testingbot.com# "Edge 83") [81](https://testingbot.com# "Edge 81") [80](https://testingbot.com# "Edge 80") ![opera](https://testingbot.com/assets/environments/svg/opera-fe6e1d716472033125a969dfe6489ab45f0eded91e9260292d6c4ce9b682a330.svg "Opera") ### Opera [122](https://testingbot.com# "Opera 122") [121](https://testingbot.com# "Opera 121") [120](https://testingbot.com# "Opera 120") [119](https://testingbot.com# "Opera 119") [118](https://testingbot.com# "Opera 118") [117](https://testingbot.com# "Opera 117") [116](https://testingbot.com# "Opera 116") [115](https://testingbot.com# "Opera 115") [114](https://testingbot.com# "Opera 114") [113](https://testingbot.com# "Opera 113") [112](https://testingbot.com# "Opera 112") [111](https://testingbot.com# "Opera 111") [110](https://testingbot.com# "Opera 110") [109](https://testingbot.com# "Opera 109") [108](https://testingbot.com# "Opera 108") [107](https://testingbot.com# "Opera 107") [106](https://testingbot.com# "Opera 106") [105](https://testingbot.com# "Opera 105") [104](https://testingbot.com# "Opera 104") [103](https://testingbot.com# "Opera 103") [102](https://testingbot.com# "Opera 102") [101](https://testingbot.com# "Opera 101") [100](https://testingbot.com# "Opera 100") [99](https://testingbot.com# "Opera 99") [98](https://testingbot.com# "Opera 98") [97](https://testingbot.com# "Opera 97") [96](https://testingbot.com# "Opera 96") [95](https://testingbot.com# "Opera 95") [94](https://testingbot.com# "Opera 94") [93](https://testingbot.com# "Opera 93") [92](https://testingbot.com# "Opera 92") [91](https://testingbot.com# "Opera 91") [90](https://testingbot.com# "Opera 90") [89](https://testingbot.com# "Opera 89") [88](https://testingbot.com# "Opera 88") [87](https://testingbot.com# "Opera 87") [86](https://testingbot.com# "Opera 86") [85](https://testingbot.com# "Opera 85") [84](https://testingbot.com# "Opera 84") [83](https://testingbot.com# "Opera 83") [82](https://testingbot.com# "Opera 82") [81](https://testingbot.com# "Opera 81") [80](https://testingbot.com# "Opera 80") [79](https://testingbot.com# "Opera 79") [78](https://testingbot.com# "Opera 78") [77](https://testingbot.com# "Opera 77") [76](https://testingbot.com# "Opera 76") [75](https://testingbot.com# "Opera 75") [74](https://testingbot.com# "Opera 74") [73](https://testingbot.com# "Opera 73") [72](https://testingbot.com# "Opera 72") [71](https://testingbot.com# "Opera 71") [70](https://testingbot.com# "Opera 70") [69](https://testingbot.com# "Opera 69") [68](https://testingbot.com# "Opera 68") [67](https://testingbot.com# "Opera 67") [66](https://testingbot.com# "Opera 66") [65](https://testingbot.com# "Opera 65") [64](https://testingbot.com# "Opera 64") [63](https://testingbot.com# "Opera 63") [62](https://testingbot.com# "Opera 62") [60](https://testingbot.com# "Opera 60") [59](https://testingbot.com# "Opera 59") [58](https://testingbot.com# "Opera 58") [57](https://testingbot.com# "Opera 57") [56](https://testingbot.com# "Opera 56") [55](https://testingbot.com# "Opera 55") [54](https://testingbot.com# "Opera 54") [53](https://testingbot.com# "Opera 53") [52](https://testingbot.com# "Opera 52") [51](https://testingbot.com# "Opera 51") [50](https://testingbot.com# "Opera 50") ![safari](https://testingbot.com/assets/environments/svg/safari-e4837bb5f9a9729fef843af4180372e975c62587808b8ab479f615f3e240e858.svg "Safari") ### Safari [14](https://testingbot.com# "Safari 14") ![macOS Catalina Browser Testing](https://testingbot.com/assets/environments/catalina_logo-7e6a89b3b4f29856729312cc930ad86462d2ad73b3aef4c7a150b5214e85f688.png) ## macOS Catalina (10.15) ![googlechrome](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg "Googlechrome") ### Chrome [beta](https://testingbot.com# "Chrome") [dev](https://testingbot.com# "Chrome") [128](https://testingbot.com# "Chrome 128") [127](https://testingbot.com# "Chrome 127") [126](https://testingbot.com# "Chrome 126") [125](https://testingbot.com# "Chrome 125") [124](https://testingbot.com# "Chrome 124") [123](https://testingbot.com# "Chrome 123") [122](https://testingbot.com# "Chrome 122") [121](https://testingbot.com# "Chrome 121") [120](https://testingbot.com# "Chrome 120") [119](https://testingbot.com# "Chrome 119") [118](https://testingbot.com# "Chrome 118") [117](https://testingbot.com# "Chrome 117") [116](https://testingbot.com# "Chrome 116") [115](https://testingbot.com# "Chrome 115") [114](https://testingbot.com# "Chrome 114") [113](https://testingbot.com# "Chrome 113") [112](https://testingbot.com# "Chrome 112") [111](https://testingbot.com# "Chrome 111") [110](https://testingbot.com# "Chrome 110") [109](https://testingbot.com# "Chrome 109") [108](https://testingbot.com# "Chrome 108") [107](https://testingbot.com# "Chrome 107") [106](https://testingbot.com# "Chrome 106") [105](https://testingbot.com# "Chrome 105") [104](https://testingbot.com# "Chrome 104") [103](https://testingbot.com# "Chrome 103") [102](https://testingbot.com# "Chrome 102") [101](https://testingbot.com# "Chrome 101") [100](https://testingbot.com# "Chrome 100") [99](https://testingbot.com# "Chrome 99") [98](https://testingbot.com# "Chrome 98") [97](https://testingbot.com# "Chrome 97") [96](https://testingbot.com# "Chrome 96") [95](https://testingbot.com# "Chrome 95") [94](https://testingbot.com# "Chrome 94") [93](https://testingbot.com# "Chrome 93") [92](https://testingbot.com# "Chrome 92") [91](https://testingbot.com# "Chrome 91") [90](https://testingbot.com# "Chrome 90") [89](https://testingbot.com# "Chrome 89") [88](https://testingbot.com# "Chrome 88") [87](https://testingbot.com# "Chrome 87") [86](https://testingbot.com# "Chrome 86") [85](https://testingbot.com# "Chrome 85") [84](https://testingbot.com# "Chrome 84") [83](https://testingbot.com# "Chrome 83") [81](https://testingbot.com# "Chrome 81") [80](https://testingbot.com# "Chrome 80") [79](https://testingbot.com# "Chrome 79") [78](https://testingbot.com# "Chrome 78") [77](https://testingbot.com# "Chrome 77") [76](https://testingbot.com# "Chrome 76") [75](https://testingbot.com# "Chrome 75") [74](https://testingbot.com# "Chrome 74") [73](https://testingbot.com# "Chrome 73") [72](https://testingbot.com# "Chrome 72") [71](https://testingbot.com# "Chrome 71") [70](https://testingbot.com# "Chrome 70") [69](https://testingbot.com# "Chrome 69") [68](https://testingbot.com# "Chrome 68") [67](https://testingbot.com# "Chrome 67") [66](https://testingbot.com# "Chrome 66") [65](https://testingbot.com# "Chrome 65") [64](https://testingbot.com# "Chrome 64") [63](https://testingbot.com# "Chrome 63") [62](https://testingbot.com# "Chrome 62") [61](https://testingbot.com# "Chrome 61") [60](https://testingbot.com# "Chrome 60") [59](https://testingbot.com# "Chrome 59") [58](https://testingbot.com# "Chrome 58") [57](https://testingbot.com# "Chrome 57") [56](https://testingbot.com# "Chrome 56") [55](https://testingbot.com# "Chrome 55") [54](https://testingbot.com# "Chrome 54") [53](https://testingbot.com# "Chrome 53") [52](https://testingbot.com# "Chrome 52") [51](https://testingbot.com# "Chrome 51") [50](https://testingbot.com# "Chrome 50") [49](https://testingbot.com# "Chrome 49") [48](https://testingbot.com# "Chrome 48") [47](https://testingbot.com# "Chrome 47") [46](https://testingbot.com# "Chrome 46") [45](https://testingbot.com# "Chrome 45") [44](https://testingbot.com# "Chrome 44") [43](https://testingbot.com# "Chrome 43") [42](https://testingbot.com# "Chrome 42") [41](https://testingbot.com# "Chrome 41") [40](https://testingbot.com# "Chrome 40") [39](https://testingbot.com# "Chrome 39") [38](https://testingbot.com# "Chrome 38") [36](https://testingbot.com# "Chrome 36") ![firefox](https://testingbot.com/assets/environments/svg/firefox-90dce8de549728e54bd3209cfacb9f1a868c2c8e70d243eb5b930d69cbb2fb1e.svg "Firefox") ### Firefox [dev](https://testingbot.com# "FF dev") [beta](https://testingbot.com# "FF beta") [155](https://testingbot.com# "FF 155") [154](https://testingbot.com# "FF 154") [153](https://testingbot.com# "FF 153") [152](https://testingbot.com# "FF 152") [151](https://testingbot.com# "FF 151") [150](https://testingbot.com# "FF 150") [149](https://testingbot.com# "FF 149") [148](https://testingbot.com# "FF 148") [147](https://testingbot.com# "FF 147") [146](https://testingbot.com# "FF 146") [145](https://testingbot.com# "FF 145") [144](https://testingbot.com# "FF 144") [143](https://testingbot.com# "FF 143") [142](https://testingbot.com# "FF 142") [141](https://testingbot.com# "FF 141") [140](https://testingbot.com# "FF 140") [139](https://testingbot.com# "FF 139") [138](https://testingbot.com# "FF 138") [137](https://testingbot.com# "FF 137") [136](https://testingbot.com# "FF 136") [135](https://testingbot.com# "FF 135") [134](https://testingbot.com# "FF 134") [133](https://testingbot.com# "FF 133") [132](https://testingbot.com# "FF 132") [131](https://testingbot.com# "FF 131") [130](https://testingbot.com# "FF 130") [129](https://testingbot.com# "FF 129") [128](https://testingbot.com# "FF 128") [127](https://testingbot.com# "FF 127") [126](https://testingbot.com# "FF 126") [125](https://testingbot.com# "FF 125") [124](https://testingbot.com# "FF 124") [123](https://testingbot.com# "FF 123") [122](https://testingbot.com# "FF 122") [121](https://testingbot.com# "FF 121") [120](https://testingbot.com# "FF 120") [119](https://testingbot.com# "FF 119") [118](https://testingbot.com# "FF 118") [117](https://testingbot.com# "FF 117") [116](https://testingbot.com# "FF 116") [115](https://testingbot.com# "FF 115") [114](https://testingbot.com# "FF 114") [113](https://testingbot.com# "FF 113") [112](https://testingbot.com# "FF 112") [111](https://testingbot.com# "FF 111") [110](https://testingbot.com# "FF 110") [109](https://testingbot.com# "FF 109") [108](https://testingbot.com# "FF 108") [107](https://testingbot.com# "FF 107") [106](https://testingbot.com# "FF 106") [105](https://testingbot.com# "FF 105") [104](https://testingbot.com# "FF 104") [103](https://testingbot.com# "FF 103") [102](https://testingbot.com# "FF 102") [101](https://testingbot.com# "FF 101") [100](https://testingbot.com# "FF 100") [99](https://testingbot.com# "FF 99") [98](https://testingbot.com# "FF 98") [97](https://testingbot.com# "FF 97") [96](https://testingbot.com# "FF 96") [95](https://testingbot.com# "FF 95") [94](https://testingbot.com# "FF 94") [93](https://testingbot.com# "FF 93") [92](https://testingbot.com# "FF 92") [91](https://testingbot.com# "FF 91") [90](https://testingbot.com# "FF 90") [89](https://testingbot.com# "FF 89") [88](https://testingbot.com# "FF 88") [87](https://testingbot.com# "FF 87") [86](https://testingbot.com# "FF 86") [85](https://testingbot.com# "FF 85") [84](https://testingbot.com# "FF 84") [83](https://testingbot.com# "FF 83") [82](https://testingbot.com# "FF 82") [81](https://testingbot.com# "FF 81") [80](https://testingbot.com# "FF 80") [79](https://testingbot.com# "FF 79") [78](https://testingbot.com# "FF 78") [77](https://testingbot.com# "FF 77") [76](https://testingbot.com# "FF 76") [75](https://testingbot.com# "FF 75") [74](https://testingbot.com# "FF 74") [73](https://testingbot.com# "FF 73") [72](https://testingbot.com# "FF 72") [71](https://testingbot.com# "FF 71") [70](https://testingbot.com# "FF 70") [69](https://testingbot.com# "FF 69") [68](https://testingbot.com# "FF 68") [67](https://testingbot.com# "FF 67") [66](https://testingbot.com# "FF 66") [65](https://testingbot.com# "FF 65") [64](https://testingbot.com# "FF 64") [63](https://testingbot.com# "FF 63") [62](https://testingbot.com# "FF 62") [61](https://testingbot.com# "FF 61") [60](https://testingbot.com# "FF 60") [59](https://testingbot.com# "FF 59") [58](https://testingbot.com# "FF 58") [57](https://testingbot.com# "FF 57") [56](https://testingbot.com# "FF 56") [55](https://testingbot.com# "FF 55") [54](https://testingbot.com# "FF 54") [53](https://testingbot.com# "FF 53") [52](https://testingbot.com# "FF 52") [51](https://testingbot.com# "FF 51") [50](https://testingbot.com# "FF 50") [49](https://testingbot.com# "FF 49") [48](https://testingbot.com# "FF 48") [47](https://testingbot.com# "FF 47") [46](https://testingbot.com# "FF 46") [45](https://testingbot.com# "FF 45") [44](https://testingbot.com# "FF 44") [43](https://testingbot.com# "FF 43") [42](https://testingbot.com# "FF 42") [41](https://testingbot.com# "FF 41") [40](https://testingbot.com# "FF 40") [39](https://testingbot.com# "FF 39") [38](https://testingbot.com# "FF 38") [37](https://testingbot.com# "FF 37") [36](https://testingbot.com# "FF 36") [35](https://testingbot.com# "FF 35") [34](https://testingbot.com# "FF 34") [33](https://testingbot.com# "FF 33") [32](https://testingbot.com# "FF 32") [31](https://testingbot.com# "FF 31") [30](https://testingbot.com# "FF 30") [29](https://testingbot.com# "FF 29") [28](https://testingbot.com# "FF 28") [27](https://testingbot.com# "FF 27") [26](https://testingbot.com# "FF 26") [25](https://testingbot.com# "FF 25") [24](https://testingbot.com# "FF 24") [23](https://testingbot.com# "FF 23") [22](https://testingbot.com# "FF 22") [21](https://testingbot.com# "FF 21") [20](https://testingbot.com# "FF 20") [19](https://testingbot.com# "FF 19") [18](https://testingbot.com# "FF 18") [17](https://testingbot.com# "FF 17") [16](https://testingbot.com# "FF 16") [15](https://testingbot.com# "FF 15") [14](https://testingbot.com# "FF 14") [13](https://testingbot.com# "FF 13") [12](https://testingbot.com# "FF 12") [11](https://testingbot.com# "FF 11") [10](https://testingbot.com# "FF 10") [9](https://testingbot.com# "FF 9") [8](https://testingbot.com# "FF 8") [7](https://testingbot.com# "FF 7") [6](https://testingbot.com# "FF 6") [5](https://testingbot.com# "FF 5") [4](https://testingbot.com# "FF 4") ![microsoftedge](https://testingbot.com/assets/environments/svg/edge-b639cac0c7a843271a0d3a6ccf894cc296064b63b2cd2920209bc5d030f3c70b.svg "Microsoftedge") ### Edge [122](https://testingbot.com# "Edge 122") [121](https://testingbot.com# "Edge 121") [120](https://testingbot.com# "Edge 120") [119](https://testingbot.com# "Edge 119") [118](https://testingbot.com# "Edge 118") [117](https://testingbot.com# "Edge 117") [116](https://testingbot.com# "Edge 116") [115](https://testingbot.com# "Edge 115") [114](https://testingbot.com# "Edge 114") [113](https://testingbot.com# "Edge 113") [112](https://testingbot.com# "Edge 112") [111](https://testingbot.com# "Edge 111") [110](https://testingbot.com# "Edge 110") [109](https://testingbot.com# "Edge 109") [108](https://testingbot.com# "Edge 108") [107](https://testingbot.com# "Edge 107") [106](https://testingbot.com# "Edge 106") [105](https://testingbot.com# "Edge 105") [104](https://testingbot.com# "Edge 104") [103](https://testingbot.com# "Edge 103") [102](https://testingbot.com# "Edge 102") [101](https://testingbot.com# "Edge 101") [100](https://testingbot.com# "Edge 100") [99](https://testingbot.com# "Edge 99") [98](https://testingbot.com# "Edge 98") [97](https://testingbot.com# "Edge 97") [96](https://testingbot.com# "Edge 96") [95](https://testingbot.com# "Edge 95") [94](https://testingbot.com# "Edge 94") [93](https://testingbot.com# "Edge 93") [92](https://testingbot.com# "Edge 92") [91](https://testingbot.com# "Edge 91") [90](https://testingbot.com# "Edge 90") [89](https://testingbot.com# "Edge 89") [88](https://testingbot.com# "Edge 88") [87](https://testingbot.com# "Edge 87") [86](https://testingbot.com# "Edge 86") [85](https://testingbot.com# "Edge 85") [84](https://testingbot.com# "Edge 84") [83](https://testingbot.com# "Edge 83") [81](https://testingbot.com# "Edge 81") [80](https://testingbot.com# "Edge 80") ![opera](https://testingbot.com/assets/environments/svg/opera-fe6e1d716472033125a969dfe6489ab45f0eded91e9260292d6c4ce9b682a330.svg "Opera") ### Opera [129](https://testingbot.com# "Opera 129") [128](https://testingbot.com# "Opera 128") [127](https://testingbot.com# "Opera 127") [126](https://testingbot.com# "Opera 126") [125](https://testingbot.com# "Opera 125") [124](https://testingbot.com# "Opera 124") [123](https://testingbot.com# "Opera 123") [122](https://testingbot.com# "Opera 122") [121](https://testingbot.com# "Opera 121") [120](https://testingbot.com# "Opera 120") [119](https://testingbot.com# "Opera 119") [118](https://testingbot.com# "Opera 118") [117](https://testingbot.com# "Opera 117") [116](https://testingbot.com# "Opera 116") [115](https://testingbot.com# "Opera 115") [114](https://testingbot.com# "Opera 114") [113](https://testingbot.com# "Opera 113") [112](https://testingbot.com# "Opera 112") [111](https://testingbot.com# "Opera 111") [110](https://testingbot.com# "Opera 110") [109](https://testingbot.com# "Opera 109") [108](https://testingbot.com# "Opera 108") [107](https://testingbot.com# "Opera 107") [106](https://testingbot.com# "Opera 106") [105](https://testingbot.com# "Opera 105") [104](https://testingbot.com# "Opera 104") [103](https://testingbot.com# "Opera 103") [102](https://testingbot.com# "Opera 102") [101](https://testingbot.com# "Opera 101") [100](https://testingbot.com# "Opera 100") [99](https://testingbot.com# "Opera 99") [98](https://testingbot.com# "Opera 98") [97](https://testingbot.com# "Opera 97") [96](https://testingbot.com# "Opera 96") [95](https://testingbot.com# "Opera 95") [94](https://testingbot.com# "Opera 94") [93](https://testingbot.com# "Opera 93") [92](https://testingbot.com# "Opera 92") [91](https://testingbot.com# "Opera 91") [90](https://testingbot.com# "Opera 90") [89](https://testingbot.com# "Opera 89") [88](https://testingbot.com# "Opera 88") [87](https://testingbot.com# "Opera 87") [86](https://testingbot.com# "Opera 86") [85](https://testingbot.com# "Opera 85") [84](https://testingbot.com# "Opera 84") [83](https://testingbot.com# "Opera 83") [82](https://testingbot.com# "Opera 82") [81](https://testingbot.com# "Opera 81") [80](https://testingbot.com# "Opera 80") [79](https://testingbot.com# "Opera 79") [78](https://testingbot.com# "Opera 78") [77](https://testingbot.com# "Opera 77") [76](https://testingbot.com# "Opera 76") [75](https://testingbot.com# "Opera 75") [74](https://testingbot.com# "Opera 74") [73](https://testingbot.com# "Opera 73") [72](https://testingbot.com# "Opera 72") [71](https://testingbot.com# "Opera 71") [70](https://testingbot.com# "Opera 70") [69](https://testingbot.com# "Opera 69") [68](https://testingbot.com# "Opera 68") [67](https://testingbot.com# "Opera 67") [66](https://testingbot.com# "Opera 66") [65](https://testingbot.com# "Opera 65") [64](https://testingbot.com# "Opera 64") [63](https://testingbot.com# "Opera 63") [62](https://testingbot.com# "Opera 62") [60](https://testingbot.com# "Opera 60") [59](https://testingbot.com# "Opera 59") [58](https://testingbot.com# "Opera 58") [57](https://testingbot.com# "Opera 57") [56](https://testingbot.com# "Opera 56") [55](https://testingbot.com# "Opera 55") [54](https://testingbot.com# "Opera 54") [53](https://testingbot.com# "Opera 53") [52](https://testingbot.com# "Opera 52") [51](https://testingbot.com# "Opera 51") [50](https://testingbot.com# "Opera 50") ![safari](https://testingbot.com/assets/environments/svg/safari-e4837bb5f9a9729fef843af4180372e975c62587808b8ab479f615f3e240e858.svg "Safari") ### Safari [13](https://testingbot.com# "Safari 13") ![macOS Mojave Browser Testing](https://testingbot.com/assets/environments/mojave_logo-55d4c0e36e23d7b1a7bc9b347619466a3311dda5b6e116385afdf504363c5d91.png) ## macOS Mojave (10.14) ![googlechrome](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg "Googlechrome") ### Chrome [116](https://testingbot.com# "Chrome 116") [115](https://testingbot.com# "Chrome 115") [114](https://testingbot.com# "Chrome 114") [113](https://testingbot.com# "Chrome 113") [112](https://testingbot.com# "Chrome 112") [111](https://testingbot.com# "Chrome 111") [110](https://testingbot.com# "Chrome 110") [109](https://testingbot.com# "Chrome 109") [108](https://testingbot.com# "Chrome 108") [107](https://testingbot.com# "Chrome 107") [106](https://testingbot.com# "Chrome 106") [105](https://testingbot.com# "Chrome 105") [104](https://testingbot.com# "Chrome 104") [103](https://testingbot.com# "Chrome 103") [102](https://testingbot.com# "Chrome 102") [101](https://testingbot.com# "Chrome 101") [100](https://testingbot.com# "Chrome 100") [99](https://testingbot.com# "Chrome 99") [98](https://testingbot.com# "Chrome 98") [97](https://testingbot.com# "Chrome 97") [96](https://testingbot.com# "Chrome 96") [95](https://testingbot.com# "Chrome 95") [94](https://testingbot.com# "Chrome 94") [93](https://testingbot.com# "Chrome 93") [92](https://testingbot.com# "Chrome 92") [91](https://testingbot.com# "Chrome 91") [90](https://testingbot.com# "Chrome 90") [89](https://testingbot.com# "Chrome 89") [88](https://testingbot.com# "Chrome 88") [87](https://testingbot.com# "Chrome 87") [86](https://testingbot.com# "Chrome 86") [85](https://testingbot.com# "Chrome 85") [84](https://testingbot.com# "Chrome 84") [83](https://testingbot.com# "Chrome 83") [81](https://testingbot.com# "Chrome 81") [80](https://testingbot.com# "Chrome 80") [79](https://testingbot.com# "Chrome 79") [78](https://testingbot.com# "Chrome 78") [77](https://testingbot.com# "Chrome 77") [76](https://testingbot.com# "Chrome 76") [75](https://testingbot.com# "Chrome 75") [74](https://testingbot.com# "Chrome 74") [73](https://testingbot.com# "Chrome 73") [72](https://testingbot.com# "Chrome 72") [71](https://testingbot.com# "Chrome 71") [70](https://testingbot.com# "Chrome 70") [69](https://testingbot.com# "Chrome 69") [68](https://testingbot.com# "Chrome 68") [67](https://testingbot.com# "Chrome 67") [66](https://testingbot.com# "Chrome 66") [65](https://testingbot.com# "Chrome 65") [64](https://testingbot.com# "Chrome 64") [63](https://testingbot.com# "Chrome 63") [62](https://testingbot.com# "Chrome 62") [61](https://testingbot.com# "Chrome 61") [60](https://testingbot.com# "Chrome 60") [59](https://testingbot.com# "Chrome 59") [58](https://testingbot.com# "Chrome 58") [57](https://testingbot.com# "Chrome 57") [56](https://testingbot.com# "Chrome 56") [55](https://testingbot.com# "Chrome 55") [54](https://testingbot.com# "Chrome 54") [53](https://testingbot.com# "Chrome 53") [52](https://testingbot.com# "Chrome 52") [51](https://testingbot.com# "Chrome 51") [50](https://testingbot.com# "Chrome 50") [49](https://testingbot.com# "Chrome 49") [48](https://testingbot.com# "Chrome 48") [47](https://testingbot.com# "Chrome 47") [46](https://testingbot.com# "Chrome 46") [45](https://testingbot.com# "Chrome 45") [44](https://testingbot.com# "Chrome 44") [43](https://testingbot.com# "Chrome 43") [42](https://testingbot.com# "Chrome 42") [41](https://testingbot.com# "Chrome 41") [40](https://testingbot.com# "Chrome 40") [39](https://testingbot.com# "Chrome 39") [38](https://testingbot.com# "Chrome 38") [36](https://testingbot.com# "Chrome 36") ![firefox](https://testingbot.com/assets/environments/svg/firefox-90dce8de549728e54bd3209cfacb9f1a868c2c8e70d243eb5b930d69cbb2fb1e.svg "Firefox") ### Firefox [115](https://testingbot.com# "FF 115") [114](https://testingbot.com# "FF 114") [113](https://testingbot.com# "FF 113") [112](https://testingbot.com# "FF 112") [111](https://testingbot.com# "FF 111") [110](https://testingbot.com# "FF 110") [109](https://testingbot.com# "FF 109") [108](https://testingbot.com# "FF 108") [107](https://testingbot.com# "FF 107") [106](https://testingbot.com# "FF 106") [105](https://testingbot.com# "FF 105") [104](https://testingbot.com# "FF 104") [103](https://testingbot.com# "FF 103") [102](https://testingbot.com# "FF 102") [101](https://testingbot.com# "FF 101") [100](https://testingbot.com# "FF 100") [99](https://testingbot.com# "FF 99") [98](https://testingbot.com# "FF 98") [97](https://testingbot.com# "FF 97") [96](https://testingbot.com# "FF 96") [95](https://testingbot.com# "FF 95") [94](https://testingbot.com# "FF 94") [93](https://testingbot.com# "FF 93") [92](https://testingbot.com# "FF 92") [91](https://testingbot.com# "FF 91") [90](https://testingbot.com# "FF 90") [89](https://testingbot.com# "FF 89") [88](https://testingbot.com# "FF 88") [87](https://testingbot.com# "FF 87") [86](https://testingbot.com# "FF 86") [85](https://testingbot.com# "FF 85") [84](https://testingbot.com# "FF 84") [83](https://testingbot.com# "FF 83") [82](https://testingbot.com# "FF 82") [81](https://testingbot.com# "FF 81") [80](https://testingbot.com# "FF 80") [79](https://testingbot.com# "FF 79") [78](https://testingbot.com# "FF 78") [77](https://testingbot.com# "FF 77") [76](https://testingbot.com# "FF 76") [75](https://testingbot.com# "FF 75") [74](https://testingbot.com# "FF 74") [73](https://testingbot.com# "FF 73") [72](https://testingbot.com# "FF 72") [71](https://testingbot.com# "FF 71") [70](https://testingbot.com# "FF 70") [69](https://testingbot.com# "FF 69") [68](https://testingbot.com# "FF 68") [67](https://testingbot.com# "FF 67") [66](https://testingbot.com# "FF 66") [65](https://testingbot.com# "FF 65") [64](https://testingbot.com# "FF 64") [63](https://testingbot.com# "FF 63") [62](https://testingbot.com# "FF 62") [61](https://testingbot.com# "FF 61") [60](https://testingbot.com# "FF 60") [59](https://testingbot.com# "FF 59") [58](https://testingbot.com# "FF 58") [57](https://testingbot.com# "FF 57") [56](https://testingbot.com# "FF 56") [55](https://testingbot.com# "FF 55") [54](https://testingbot.com# "FF 54") [53](https://testingbot.com# "FF 53") [52](https://testingbot.com# "FF 52") [51](https://testingbot.com# "FF 51") [50](https://testingbot.com# "FF 50") [49](https://testingbot.com# "FF 49") [48](https://testingbot.com# "FF 48") [47](https://testingbot.com# "FF 47") [46](https://testingbot.com# "FF 46") [45](https://testingbot.com# "FF 45") [44](https://testingbot.com# "FF 44") [43](https://testingbot.com# "FF 43") [42](https://testingbot.com# "FF 42") [41](https://testingbot.com# "FF 41") [40](https://testingbot.com# "FF 40") [39](https://testingbot.com# "FF 39") [38](https://testingbot.com# "FF 38") [37](https://testingbot.com# "FF 37") [36](https://testingbot.com# "FF 36") [35](https://testingbot.com# "FF 35") [34](https://testingbot.com# "FF 34") [33](https://testingbot.com# "FF 33") [32](https://testingbot.com# "FF 32") [31](https://testingbot.com# "FF 31") [30](https://testingbot.com# "FF 30") [29](https://testingbot.com# "FF 29") [28](https://testingbot.com# "FF 28") [27](https://testingbot.com# "FF 27") [26](https://testingbot.com# "FF 26") [25](https://testingbot.com# "FF 25") [24](https://testingbot.com# "FF 24") [23](https://testingbot.com# "FF 23") [22](https://testingbot.com# "FF 22") [21](https://testingbot.com# "FF 21") [20](https://testingbot.com# "FF 20") [19](https://testingbot.com# "FF 19") [18](https://testingbot.com# "FF 18") [17](https://testingbot.com# "FF 17") [16](https://testingbot.com# "FF 16") [15](https://testingbot.com# "FF 15") [14](https://testingbot.com# "FF 14") [13](https://testingbot.com# "FF 13") [12](https://testingbot.com# "FF 12") [11](https://testingbot.com# "FF 11") [10](https://testingbot.com# "FF 10") [9](https://testingbot.com# "FF 9") [8](https://testingbot.com# "FF 8") [7](https://testingbot.com# "FF 7") [6](https://testingbot.com# "FF 6") [5](https://testingbot.com# "FF 5") [4](https://testingbot.com# "FF 4") ![microsoftedge](https://testingbot.com/assets/environments/svg/edge-b639cac0c7a843271a0d3a6ccf894cc296064b63b2cd2920209bc5d030f3c70b.svg "Microsoftedge") ### Edge [116](https://testingbot.com# "Edge 116") [115](https://testingbot.com# "Edge 115") [114](https://testingbot.com# "Edge 114") [113](https://testingbot.com# "Edge 113") [112](https://testingbot.com# "Edge 112") [111](https://testingbot.com# "Edge 111") [110](https://testingbot.com# "Edge 110") [109](https://testingbot.com# "Edge 109") [108](https://testingbot.com# "Edge 108") [107](https://testingbot.com# "Edge 107") [106](https://testingbot.com# "Edge 106") [105](https://testingbot.com# "Edge 105") [104](https://testingbot.com# "Edge 104") [103](https://testingbot.com# "Edge 103") [102](https://testingbot.com# "Edge 102") [101](https://testingbot.com# "Edge 101") [100](https://testingbot.com# "Edge 100") [99](https://testingbot.com# "Edge 99") [98](https://testingbot.com# "Edge 98") [97](https://testingbot.com# "Edge 97") [96](https://testingbot.com# "Edge 96") [95](https://testingbot.com# "Edge 95") [94](https://testingbot.com# "Edge 94") [93](https://testingbot.com# "Edge 93") [92](https://testingbot.com# "Edge 92") [91](https://testingbot.com# "Edge 91") [90](https://testingbot.com# "Edge 90") [89](https://testingbot.com# "Edge 89") [88](https://testingbot.com# "Edge 88") [87](https://testingbot.com# "Edge 87") [86](https://testingbot.com# "Edge 86") [85](https://testingbot.com# "Edge 85") [84](https://testingbot.com# "Edge 84") [83](https://testingbot.com# "Edge 83") [81](https://testingbot.com# "Edge 81") [80](https://testingbot.com# "Edge 80") [79](https://testingbot.com# "Edge 79") ![opera](https://testingbot.com/assets/environments/svg/opera-fe6e1d716472033125a969dfe6489ab45f0eded91e9260292d6c4ce9b682a330.svg "Opera") ### Opera [102](https://testingbot.com# "Opera 102") [101](https://testingbot.com# "Opera 101") [100](https://testingbot.com# "Opera 100") [99](https://testingbot.com# "Opera 99") [98](https://testingbot.com# "Opera 98") [97](https://testingbot.com# "Opera 97") [96](https://testingbot.com# "Opera 96") [95](https://testingbot.com# "Opera 95") [94](https://testingbot.com# "Opera 94") [93](https://testingbot.com# "Opera 93") [92](https://testingbot.com# "Opera 92") [91](https://testingbot.com# "Opera 91") [90](https://testingbot.com# "Opera 90") [89](https://testingbot.com# "Opera 89") [88](https://testingbot.com# "Opera 88") [87](https://testingbot.com# "Opera 87") [86](https://testingbot.com# "Opera 86") [85](https://testingbot.com# "Opera 85") [84](https://testingbot.com# "Opera 84") [83](https://testingbot.com# "Opera 83") [82](https://testingbot.com# "Opera 82") [81](https://testingbot.com# "Opera 81") [80](https://testingbot.com# "Opera 80") [79](https://testingbot.com# "Opera 79") [78](https://testingbot.com# "Opera 78") [77](https://testingbot.com# "Opera 77") [76](https://testingbot.com# "Opera 76") [75](https://testingbot.com# "Opera 75") [74](https://testingbot.com# "Opera 74") [73](https://testingbot.com# "Opera 73") [72](https://testingbot.com# "Opera 72") [71](https://testingbot.com# "Opera 71") [70](https://testingbot.com# "Opera 70") [69](https://testingbot.com# "Opera 69") [68](https://testingbot.com# "Opera 68") [67](https://testingbot.com# "Opera 67") [66](https://testingbot.com# "Opera 66") [65](https://testingbot.com# "Opera 65") [64](https://testingbot.com# "Opera 64") [63](https://testingbot.com# "Opera 63") [62](https://testingbot.com# "Opera 62") [60](https://testingbot.com# "Opera 60") [59](https://testingbot.com# "Opera 59") [58](https://testingbot.com# "Opera 58") [57](https://testingbot.com# "Opera 57") [56](https://testingbot.com# "Opera 56") [55](https://testingbot.com# "Opera 55") [54](https://testingbot.com# "Opera 54") [53](https://testingbot.com# "Opera 53") [52](https://testingbot.com# "Opera 52") [51](https://testingbot.com# "Opera 51") [50](https://testingbot.com# "Opera 50") ![safari](https://testingbot.com/assets/environments/svg/safari-e4837bb5f9a9729fef843af4180372e975c62587808b8ab479f615f3e240e858.svg "Safari") ### Safari [12](https://testingbot.com# "Safari 12") ![macOS High-Sierra Browser Testing](https://testingbot.com/assets/environments/highsierra_logo-84730fb1f25dbf4fe5300d52b31494d8009d15b26ea893a5436040d58f19d2df.png) ## macOS High Sierra (10.13) ![googlechrome](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg "Googlechrome") ### Chrome [116](https://testingbot.com# "Chrome 116") [115](https://testingbot.com# "Chrome 115") [114](https://testingbot.com# "Chrome 114") [113](https://testingbot.com# "Chrome 113") [112](https://testingbot.com# "Chrome 112") [111](https://testingbot.com# "Chrome 111") [110](https://testingbot.com# "Chrome 110") [109](https://testingbot.com# "Chrome 109") [108](https://testingbot.com# "Chrome 108") [107](https://testingbot.com# "Chrome 107") [106](https://testingbot.com# "Chrome 106") [105](https://testingbot.com# "Chrome 105") [104](https://testingbot.com# "Chrome 104") [103](https://testingbot.com# "Chrome 103") [102](https://testingbot.com# "Chrome 102") [101](https://testingbot.com# "Chrome 101") [100](https://testingbot.com# "Chrome 100") [99](https://testingbot.com# "Chrome 99") [98](https://testingbot.com# "Chrome 98") [97](https://testingbot.com# "Chrome 97") [96](https://testingbot.com# "Chrome 96") [95](https://testingbot.com# "Chrome 95") [94](https://testingbot.com# "Chrome 94") [93](https://testingbot.com# "Chrome 93") [92](https://testingbot.com# "Chrome 92") [91](https://testingbot.com# "Chrome 91") [90](https://testingbot.com# "Chrome 90") [89](https://testingbot.com# "Chrome 89") [88](https://testingbot.com# "Chrome 88") [87](https://testingbot.com# "Chrome 87") [86](https://testingbot.com# "Chrome 86") [85](https://testingbot.com# "Chrome 85") [84](https://testingbot.com# "Chrome 84") [83](https://testingbot.com# "Chrome 83") [81](https://testingbot.com# "Chrome 81") [80](https://testingbot.com# "Chrome 80") [79](https://testingbot.com# "Chrome 79") [78](https://testingbot.com# "Chrome 78") [77](https://testingbot.com# "Chrome 77") [76](https://testingbot.com# "Chrome 76") [75](https://testingbot.com# "Chrome 75") [74](https://testingbot.com# "Chrome 74") [73](https://testingbot.com# "Chrome 73") [72](https://testingbot.com# "Chrome 72") [71](https://testingbot.com# "Chrome 71") [70](https://testingbot.com# "Chrome 70") [69](https://testingbot.com# "Chrome 69") [68](https://testingbot.com# "Chrome 68") [67](https://testingbot.com# "Chrome 67") [66](https://testingbot.com# "Chrome 66") [65](https://testingbot.com# "Chrome 65") [64](https://testingbot.com# "Chrome 64") [63](https://testingbot.com# "Chrome 63") [62](https://testingbot.com# "Chrome 62") [61](https://testingbot.com# "Chrome 61") [60](https://testingbot.com# "Chrome 60") [59](https://testingbot.com# "Chrome 59") [58](https://testingbot.com# "Chrome 58") [57](https://testingbot.com# "Chrome 57") [56](https://testingbot.com# "Chrome 56") [55](https://testingbot.com# "Chrome 55") [54](https://testingbot.com# "Chrome 54") [53](https://testingbot.com# "Chrome 53") [52](https://testingbot.com# "Chrome 52") [51](https://testingbot.com# "Chrome 51") [50](https://testingbot.com# "Chrome 50") [49](https://testingbot.com# "Chrome 49") [48](https://testingbot.com# "Chrome 48") [47](https://testingbot.com# "Chrome 47") [46](https://testingbot.com# "Chrome 46") [45](https://testingbot.com# "Chrome 45") [44](https://testingbot.com# "Chrome 44") [43](https://testingbot.com# "Chrome 43") [42](https://testingbot.com# "Chrome 42") [41](https://testingbot.com# "Chrome 41") [40](https://testingbot.com# "Chrome 40") [39](https://testingbot.com# "Chrome 39") [38](https://testingbot.com# "Chrome 38") [36](https://testingbot.com# "Chrome 36") ![firefox](https://testingbot.com/assets/environments/svg/firefox-90dce8de549728e54bd3209cfacb9f1a868c2c8e70d243eb5b930d69cbb2fb1e.svg "Firefox") ### Firefox [115](https://testingbot.com# "FF 115") [114](https://testingbot.com# "FF 114") [113](https://testingbot.com# "FF 113") [112](https://testingbot.com# "FF 112") [111](https://testingbot.com# "FF 111") [110](https://testingbot.com# "FF 110") [109](https://testingbot.com# "FF 109") [108](https://testingbot.com# "FF 108") [107](https://testingbot.com# "FF 107") [106](https://testingbot.com# "FF 106") [105](https://testingbot.com# "FF 105") [104](https://testingbot.com# "FF 104") [103](https://testingbot.com# "FF 103") [102](https://testingbot.com# "FF 102") [101](https://testingbot.com# "FF 101") [100](https://testingbot.com# "FF 100") [99](https://testingbot.com# "FF 99") [98](https://testingbot.com# "FF 98") [97](https://testingbot.com# "FF 97") [96](https://testingbot.com# "FF 96") [95](https://testingbot.com# "FF 95") [94](https://testingbot.com# "FF 94") [93](https://testingbot.com# "FF 93") [92](https://testingbot.com# "FF 92") [91](https://testingbot.com# "FF 91") [90](https://testingbot.com# "FF 90") [89](https://testingbot.com# "FF 89") [88](https://testingbot.com# "FF 88") [87](https://testingbot.com# "FF 87") [86](https://testingbot.com# "FF 86") [85](https://testingbot.com# "FF 85") [84](https://testingbot.com# "FF 84") [83](https://testingbot.com# "FF 83") [82](https://testingbot.com# "FF 82") [81](https://testingbot.com# "FF 81") [80](https://testingbot.com# "FF 80") [79](https://testingbot.com# "FF 79") [78](https://testingbot.com# "FF 78") [77](https://testingbot.com# "FF 77") [76](https://testingbot.com# "FF 76") [75](https://testingbot.com# "FF 75") [74](https://testingbot.com# "FF 74") [73](https://testingbot.com# "FF 73") [72](https://testingbot.com# "FF 72") [71](https://testingbot.com# "FF 71") [70](https://testingbot.com# "FF 70") [69](https://testingbot.com# "FF 69") [68](https://testingbot.com# "FF 68") [67](https://testingbot.com# "FF 67") [66](https://testingbot.com# "FF 66") [65](https://testingbot.com# "FF 65") [64](https://testingbot.com# "FF 64") [63](https://testingbot.com# "FF 63") [62](https://testingbot.com# "FF 62") [61](https://testingbot.com# "FF 61") [60](https://testingbot.com# "FF 60") [59](https://testingbot.com# "FF 59") [58](https://testingbot.com# "FF 58") [57](https://testingbot.com# "FF 57") [56](https://testingbot.com# "FF 56") [55](https://testingbot.com# "FF 55") [54](https://testingbot.com# "FF 54") [53](https://testingbot.com# "FF 53") [52](https://testingbot.com# "FF 52") [51](https://testingbot.com# "FF 51") [50](https://testingbot.com# "FF 50") [49](https://testingbot.com# "FF 49") [48](https://testingbot.com# "FF 48") [47](https://testingbot.com# "FF 47") [46](https://testingbot.com# "FF 46") [45](https://testingbot.com# "FF 45") [44](https://testingbot.com# "FF 44") [43](https://testingbot.com# "FF 43") [42](https://testingbot.com# "FF 42") [41](https://testingbot.com# "FF 41") [40](https://testingbot.com# "FF 40") [39](https://testingbot.com# "FF 39") [38](https://testingbot.com# "FF 38") [37](https://testingbot.com# "FF 37") [36](https://testingbot.com# "FF 36") [35](https://testingbot.com# "FF 35") [34](https://testingbot.com# "FF 34") [33](https://testingbot.com# "FF 33") [32](https://testingbot.com# "FF 32") [31](https://testingbot.com# "FF 31") [30](https://testingbot.com# "FF 30") [29](https://testingbot.com# "FF 29") [28](https://testingbot.com# "FF 28") [27](https://testingbot.com# "FF 27") [26](https://testingbot.com# "FF 26") [25](https://testingbot.com# "FF 25") [24](https://testingbot.com# "FF 24") [23](https://testingbot.com# "FF 23") [22](https://testingbot.com# "FF 22") [21](https://testingbot.com# "FF 21") [20](https://testingbot.com# "FF 20") [19](https://testingbot.com# "FF 19") [18](https://testingbot.com# "FF 18") [17](https://testingbot.com# "FF 17") [16](https://testingbot.com# "FF 16") [15](https://testingbot.com# "FF 15") [14](https://testingbot.com# "FF 14") [13](https://testingbot.com# "FF 13") [12](https://testingbot.com# "FF 12") [11](https://testingbot.com# "FF 11") [10](https://testingbot.com# "FF 10") [9](https://testingbot.com# "FF 9") [8](https://testingbot.com# "FF 8") [7](https://testingbot.com# "FF 7") [6](https://testingbot.com# "FF 6") [5](https://testingbot.com# "FF 5") [4](https://testingbot.com# "FF 4") ![opera](https://testingbot.com/assets/environments/svg/opera-fe6e1d716472033125a969dfe6489ab45f0eded91e9260292d6c4ce9b682a330.svg "Opera") ### Opera [102](https://testingbot.com# "Opera 102") [101](https://testingbot.com# "Opera 101") [100](https://testingbot.com# "Opera 100") [99](https://testingbot.com# "Opera 99") [98](https://testingbot.com# "Opera 98") [97](https://testingbot.com# "Opera 97") [96](https://testingbot.com# "Opera 96") [95](https://testingbot.com# "Opera 95") [94](https://testingbot.com# "Opera 94") [93](https://testingbot.com# "Opera 93") [92](https://testingbot.com# "Opera 92") [91](https://testingbot.com# "Opera 91") [90](https://testingbot.com# "Opera 90") [89](https://testingbot.com# "Opera 89") [88](https://testingbot.com# "Opera 88") [87](https://testingbot.com# "Opera 87") [86](https://testingbot.com# "Opera 86") [85](https://testingbot.com# "Opera 85") [84](https://testingbot.com# "Opera 84") [83](https://testingbot.com# "Opera 83") [82](https://testingbot.com# "Opera 82") [81](https://testingbot.com# "Opera 81") [80](https://testingbot.com# "Opera 80") [79](https://testingbot.com# "Opera 79") [78](https://testingbot.com# "Opera 78") [77](https://testingbot.com# "Opera 77") [76](https://testingbot.com# "Opera 76") [75](https://testingbot.com# "Opera 75") [74](https://testingbot.com# "Opera 74") [73](https://testingbot.com# "Opera 73") [72](https://testingbot.com# "Opera 72") [71](https://testingbot.com# "Opera 71") [70](https://testingbot.com# "Opera 70") [69](https://testingbot.com# "Opera 69") [68](https://testingbot.com# "Opera 68") [67](https://testingbot.com# "Opera 67") [66](https://testingbot.com# "Opera 66") [65](https://testingbot.com# "Opera 65") [64](https://testingbot.com# "Opera 64") [63](https://testingbot.com# "Opera 63") [62](https://testingbot.com# "Opera 62") [60](https://testingbot.com# "Opera 60") [59](https://testingbot.com# "Opera 59") [58](https://testingbot.com# "Opera 58") [57](https://testingbot.com# "Opera 57") [56](https://testingbot.com# "Opera 56") [55](https://testingbot.com# "Opera 55") [54](https://testingbot.com# "Opera 54") [53](https://testingbot.com# "Opera 53") [52](https://testingbot.com# "Opera 52") [51](https://testingbot.com# "Opera 51") [50](https://testingbot.com# "Opera 50") ![safari](https://testingbot.com/assets/environments/svg/safari-e4837bb5f9a9729fef843af4180372e975c62587808b8ab479f615f3e240e858.svg "Safari") ### Safari [11](https://testingbot.com# "Safari 11") ![Linux Browser Testing](https://testingbot.com/assets/environments/linux_logo-3bf359ff8b03f3abf649e853e44e89515c60c7ca40a5c09e6281e3d61e2d0e16.png) ## Linux (Ubuntu 22.04) ![googlechrome](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg "Googlechrome") ### Chrome [beta](https://testingbot.com# "Chrome") [dev](https://testingbot.com# "Chrome") [153](https://testingbot.com# "Chrome 153") [152](https://testingbot.com# "Chrome 152") [151](https://testingbot.com# "Chrome 151") [150](https://testingbot.com# "Chrome 150") [149](https://testingbot.com# "Chrome 149") [148](https://testingbot.com# "Chrome 148") [147](https://testingbot.com# "Chrome 147") [146](https://testingbot.com# "Chrome 146") [145](https://testingbot.com# "Chrome 145") [144](https://testingbot.com# "Chrome 144") [143](https://testingbot.com# "Chrome 143") [142](https://testingbot.com# "Chrome 142") [141](https://testingbot.com# "Chrome 141") [140](https://testingbot.com# "Chrome 140") [139](https://testingbot.com# "Chrome 139") [138](https://testingbot.com# "Chrome 138") [137](https://testingbot.com# "Chrome 137") [136](https://testingbot.com# "Chrome 136") [135](https://testingbot.com# "Chrome 135") [134](https://testingbot.com# "Chrome 134") [133](https://testingbot.com# "Chrome 133") [132](https://testingbot.com# "Chrome 132") [131](https://testingbot.com# "Chrome 131") [130](https://testingbot.com# "Chrome 130") [129](https://testingbot.com# "Chrome 129") [128](https://testingbot.com# "Chrome 128") [127](https://testingbot.com# "Chrome 127") [126](https://testingbot.com# "Chrome 126") [125](https://testingbot.com# "Chrome 125") [124](https://testingbot.com# "Chrome 124") [123](https://testingbot.com# "Chrome 123") [122](https://testingbot.com# "Chrome 122") [121](https://testingbot.com# "Chrome 121") [120](https://testingbot.com# "Chrome 120") [119](https://testingbot.com# "Chrome 119") [118](https://testingbot.com# "Chrome 118") [117](https://testingbot.com# "Chrome 117") [116](https://testingbot.com# "Chrome 116") [115](https://testingbot.com# "Chrome 115") [114](https://testingbot.com# "Chrome 114") [113](https://testingbot.com# "Chrome 113") [112](https://testingbot.com# "Chrome 112") [111](https://testingbot.com# "Chrome 111") [110](https://testingbot.com# "Chrome 110") [109](https://testingbot.com# "Chrome 109") [108](https://testingbot.com# "Chrome 108") [107](https://testingbot.com# "Chrome 107") [106](https://testingbot.com# "Chrome 106") [105](https://testingbot.com# "Chrome 105") [104](https://testingbot.com# "Chrome 104") [103](https://testingbot.com# "Chrome 103") [102](https://testingbot.com# "Chrome 102") [101](https://testingbot.com# "Chrome 101") [100](https://testingbot.com# "Chrome 100") [99](https://testingbot.com# "Chrome 99") [98](https://testingbot.com# "Chrome 98") [97](https://testingbot.com# "Chrome 97") [96](https://testingbot.com# "Chrome 96") [95](https://testingbot.com# "Chrome 95") [94](https://testingbot.com# "Chrome 94") [93](https://testingbot.com# "Chrome 93") [92](https://testingbot.com# "Chrome 92") [91](https://testingbot.com# "Chrome 91") [90](https://testingbot.com# "Chrome 90") [89](https://testingbot.com# "Chrome 89") [88](https://testingbot.com# "Chrome 88") [87](https://testingbot.com# "Chrome 87") [86](https://testingbot.com# "Chrome 86") [85](https://testingbot.com# "Chrome 85") [84](https://testingbot.com# "Chrome 84") [83](https://testingbot.com# "Chrome 83") [81](https://testingbot.com# "Chrome 81") [80](https://testingbot.com# "Chrome 80") [79](https://testingbot.com# "Chrome 79") [78](https://testingbot.com# "Chrome 78") [77](https://testingbot.com# "Chrome 77") [76](https://testingbot.com# "Chrome 76") [75](https://testingbot.com# "Chrome 75") [74](https://testingbot.com# "Chrome 74") [73](https://testingbot.com# "Chrome 73") [72](https://testingbot.com# "Chrome 72") [71](https://testingbot.com# "Chrome 71") [70](https://testingbot.com# "Chrome 70") [69](https://testingbot.com# "Chrome 69") [68](https://testingbot.com# "Chrome 68") [67](https://testingbot.com# "Chrome 67") [66](https://testingbot.com# "Chrome 66") [65](https://testingbot.com# "Chrome 65") [64](https://testingbot.com# "Chrome 64") [63](https://testingbot.com# "Chrome 63") [62](https://testingbot.com# "Chrome 62") [61](https://testingbot.com# "Chrome 61") [60](https://testingbot.com# "Chrome 60") [59](https://testingbot.com# "Chrome 59") [58](https://testingbot.com# "Chrome 58") [57](https://testingbot.com# "Chrome 57") [56](https://testingbot.com# "Chrome 56") [55](https://testingbot.com# "Chrome 55") [54](https://testingbot.com# "Chrome 54") [53](https://testingbot.com# "Chrome 53") [52](https://testingbot.com# "Chrome 52") [51](https://testingbot.com# "Chrome 51") [50](https://testingbot.com# "Chrome 50") [49](https://testingbot.com# "Chrome 49") [48](https://testingbot.com# "Chrome 48") [47](https://testingbot.com# "Chrome 47") [46](https://testingbot.com# "Chrome 46") [45](https://testingbot.com# "Chrome 45") [44](https://testingbot.com# "Chrome 44") [43](https://testingbot.com# "Chrome 43") [42](https://testingbot.com# "Chrome 42") [41](https://testingbot.com# "Chrome 41") [40](https://testingbot.com# "Chrome 40") [39](https://testingbot.com# "Chrome 39") [38](https://testingbot.com# "Chrome 38") [37](https://testingbot.com# "Chrome 37") ![microsoftedge](https://testingbot.com/assets/environments/svg/edge-b639cac0c7a843271a0d3a6ccf894cc296064b63b2cd2920209bc5d030f3c70b.svg "Microsoftedge") ### Edge [153](https://testingbot.com# "edge150") [152](https://testingbot.com# "edge150") [151](https://testingbot.com# "edge150") [150](https://testingbot.com# "edge150") [149](https://testingbot.com# "edge149") [148](https://testingbot.com# "edge148") [147](https://testingbot.com# "edge147") [146](https://testingbot.com# "edge146") [145](https://testingbot.com# "edge145") [144](https://testingbot.com# "edge144") [143](https://testingbot.com# "edge143") ![firefox](https://testingbot.com/assets/environments/svg/firefox-90dce8de549728e54bd3209cfacb9f1a868c2c8e70d243eb5b930d69cbb2fb1e.svg "Firefox") ### Firefox [155](https://testingbot.com# "FF 155") [154](https://testingbot.com# "FF 154") [153](https://testingbot.com# "FF 153") [152](https://testingbot.com# "FF 152") [151](https://testingbot.com# "FF 151") [150](https://testingbot.com# "FF 150") [149](https://testingbot.com# "FF 149") [148](https://testingbot.com# "FF 148") [147](https://testingbot.com# "FF 147") [146](https://testingbot.com# "FF 146") [145](https://testingbot.com# "FF 145") [144](https://testingbot.com# "FF 144") [143](https://testingbot.com# "FF 143") [142](https://testingbot.com# "FF 142") [141](https://testingbot.com# "FF 141") [140](https://testingbot.com# "FF 140") [139](https://testingbot.com# "FF 139") [138](https://testingbot.com# "FF 138") [137](https://testingbot.com# "FF 137") [136](https://testingbot.com# "FF 136") [135](https://testingbot.com# "FF 135") [134](https://testingbot.com# "FF 134") [133](https://testingbot.com# "FF 133") [132](https://testingbot.com# "FF 132") [131](https://testingbot.com# "FF 131") [130](https://testingbot.com# "FF 130") [129](https://testingbot.com# "FF 129") [128](https://testingbot.com# "FF 128") [127](https://testingbot.com# "FF 127") [126](https://testingbot.com# "FF 126") [125](https://testingbot.com# "FF 125") [124](https://testingbot.com# "FF 124") [123](https://testingbot.com# "FF 123") [122](https://testingbot.com# "FF 122") [121](https://testingbot.com# "FF 121") [120](https://testingbot.com# "FF 120") [119](https://testingbot.com# "FF 119") [118](https://testingbot.com# "FF 118") [117](https://testingbot.com# "FF 117") [116](https://testingbot.com# "FF 116") [115](https://testingbot.com# "FF 115") [114](https://testingbot.com# "FF 114") [113](https://testingbot.com# "FF 113") [112](https://testingbot.com# "FF 112") [111](https://testingbot.com# "FF 111") [110](https://testingbot.com# "FF 110") [109](https://testingbot.com# "FF 109") [108](https://testingbot.com# "FF 108") [107](https://testingbot.com# "FF 107") [106](https://testingbot.com# "FF 106") [105](https://testingbot.com# "FF 105") [104](https://testingbot.com# "FF 104") [103](https://testingbot.com# "FF 103") [102](https://testingbot.com# "FF 102") [101](https://testingbot.com# "FF 101") [100](https://testingbot.com# "FF 100") [99](https://testingbot.com# "FF 99") [98](https://testingbot.com# "FF 98") [97](https://testingbot.com# "FF 97") [96](https://testingbot.com# "FF 96") [95](https://testingbot.com# "FF 95") [94](https://testingbot.com# "FF 94") [93](https://testingbot.com# "FF 93") [92](https://testingbot.com# "FF 92") [91](https://testingbot.com# "FF 91") ![opera](https://testingbot.com/assets/environments/svg/opera-fe6e1d716472033125a969dfe6489ab45f0eded91e9260292d6c4ce9b682a330.svg "Opera") ### Opera [135](https://testingbot.com# "Opera 135") [134](https://testingbot.com# "Opera 134") [130](https://testingbot.com# "Opera 130") [129](https://testingbot.com# "Opera 129") [128](https://testingbot.com# "Opera 128") [127](https://testingbot.com# "Opera 127") [126](https://testingbot.com# "Opera 126") [125](https://testingbot.com# "Opera 125") [124](https://testingbot.com# "Opera 124") [123](https://testingbot.com# "Opera 123") [122](https://testingbot.com# "Opera 122") [121](https://testingbot.com# "Opera 121") [120](https://testingbot.com# "Opera 120") [119](https://testingbot.com# "Opera 119") [118](https://testingbot.com# "Opera 118") [117](https://testingbot.com# "Opera 117") [116](https://testingbot.com# "Opera 116") [115](https://testingbot.com# "Opera 115") [114](https://testingbot.com# "Opera 114") [113](https://testingbot.com# "Opera 113") [112](https://testingbot.com# "Opera 112") [111](https://testingbot.com# "Opera 111") [110](https://testingbot.com# "Opera 110") [109](https://testingbot.com# "Opera 109") [108](https://testingbot.com# "Opera 108") [107](https://testingbot.com# "Opera 107") [106](https://testingbot.com# "Opera 106") [105](https://testingbot.com# "Opera 105") [104](https://testingbot.com# "Opera 104") [103](https://testingbot.com# "Opera 103") [102](https://testingbot.com# "Opera 102") [101](https://testingbot.com# "Opera 101") [100](https://testingbot.com# "Opera 100") [99](https://testingbot.com# "Opera 99") [98](https://testingbot.com# "Opera 98") [97](https://testingbot.com# "Opera 97") [96](https://testingbot.com# "Opera 96") [95](https://testingbot.com# "Opera 95") [94](https://testingbot.com# "Opera 94") [93](https://testingbot.com# "Opera 93") [92](https://testingbot.com# "Opera 92") [91](https://testingbot.com# "Opera 91") [90](https://testingbot.com# "Opera 90") [89](https://testingbot.com# "Opera 89") [88](https://testingbot.com# "Opera 88") [87](https://testingbot.com# "Opera 87") [86](https://testingbot.com# "Opera 86") [85](https://testingbot.com# "Opera 85") [84](https://testingbot.com# "Opera 84") [83](https://testingbot.com# "Opera 83") [82](https://testingbot.com# "Opera 82") [81](https://testingbot.com# "Opera 81") [80](https://testingbot.com# "Opera 80") [79](https://testingbot.com# "Opera 79") ![iOS Browser Testing](https://testingbot.com/assets/environments/svg/ios-383468addf160fa18d0e431f529420739d7f7d1206175f682fead627d2e99a52.svg) ## iOS Simulators ### Phones #### 27.1 [iPhone Duo (27.1)](https://testingbot.com# "iPhone Duo - 27.1") #### 27.0 [iPhone 18 Pro (27.0)](https://testingbot.com# "iPhone 18 Pro - 27.0") [iPhone 18 Pro Max (27.0)](https://testingbot.com# "iPhone 18 Pro Max - 27.0") [iPhone 17 (27.0)](https://testingbot.com# "iPhone 17 - 27.0") [iPhone 17e (27.0)](https://testingbot.com# "iPhone 17e - 27.0") [iPhone Air (27.0)](https://testingbot.com# "iPhone Air - 27.0") #### 26.3 [iPhone 17 Pro Max (26.3)](https://testingbot.com# "iPhone 17 Pro Max - 26.3") [iPhone 17 Pro (26.3)](https://testingbot.com# "iPhone 17 Pro - 26.3") [iPhone 17 (26.3)](https://testingbot.com# "iPhone 17 - 26.3") [iPhone 16e (26.3)](https://testingbot.com# "iPhone 16e - 26.3") [iPhone Air (26.3)](https://testingbot.com# "iPhone Air - 26.3") #### 26.0 [iPhone 17 Pro Max (26.0)](https://testingbot.com# "iPhone 17 Pro Max - 26.0") [iPhone 17 Pro (26.0)](https://testingbot.com# "iPhone 17 Pro - 26.0") [iPhone 17 (26.0)](https://testingbot.com# "iPhone 17 - 26.0") [iPhone 16e (26.0)](https://testingbot.com# "iPhone 16e - 26.0") [iPhone Air (26.0)](https://testingbot.com# "iPhone Air - 26.0") #### 18.6 [iPhone 16 Pro Max (18.6)](https://testingbot.com# "iPhone 16 Pro Max - 18.6") [iPhone 16 Pro (18.6)](https://testingbot.com# "iPhone 16 Pro - 18.6") [iPhone 16 Plus (18.6)](https://testingbot.com# "iPhone 16 Plus - 18.6") [iPhone 16 (18.6)](https://testingbot.com# "iPhone 16 - 18.6") [iPhone 16e (18.6)](https://testingbot.com# "iPhone 16e - 18.6") #### 18.3 [iPhone 16 (18.3)](https://testingbot.com# "iPhone 16 - 18.3") [iPhone 16e (18.3)](https://testingbot.com# "iPhone 16e - 18.3") [iPhone 16 Plus (18.3)](https://testingbot.com# "iPhone 16 Plus - 18.3") [iPhone 16 Pro (18.3)](https://testingbot.com# "iPhone 16 Pro - 18.3") [iPhone 16 Pro Max (18.3)](https://testingbot.com# "iPhone 16 Pro Max - 18.3") #### 18.0 [iPhone 16 (18.0)](https://testingbot.com# "iPhone 16 - 18.0") [iPhone 16 Plus (18.0)](https://testingbot.com# "iPhone 16 Plus - 18.0") [iPhone 16 Pro (18.0)](https://testingbot.com# "iPhone 16 Pro - 18.0") [iPhone 16 Pro Max (18.0)](https://testingbot.com# "iPhone 16 Pro Max - 18.0") [iPhone 15 (18.0)](https://testingbot.com# "iPhone 15 - 18.0") [iPhone SE (3rd generation) (18.0)](https://testingbot.com# "iPhone SE (3rd generation) - 18.0") #### 17.5 [iPhone 15 (17.5)](https://testingbot.com# "iPhone 15 - 17.5") [iPhone 15 Plus (17.5)](https://testingbot.com# "iPhone 15 Plus - 17.5") [iPhone 15 Pro (17.5)](https://testingbot.com# "iPhone 15 Pro - 17.5") [iPhone 15 Pro Max (17.5)](https://testingbot.com# "iPhone 15 Pro Max - 17.5") [iPhone SE (3rd generation) (17.5)](https://testingbot.com# "iPhone SE (3rd generation) - 17.5") #### 17.4 [iPhone 15 (17.4)](https://testingbot.com# "iPhone 15 - 17.4") [iPhone 15 Plus (17.4)](https://testingbot.com# "iPhone 15 Plus - 17.4") [iPhone 15 Pro (17.4)](https://testingbot.com# "iPhone 15 Pro - 17.4") [iPhone 15 Pro Max (17.4)](https://testingbot.com# "iPhone 15 Pro Max - 17.4") [iPhone SE (3rd generation) (17.4)](https://testingbot.com# "iPhone SE (3rd generation) - 17.4") #### 17.2 [iPhone 15 (17.2)](https://testingbot.com# "iPhone 15 - 17.2") [iPhone 15 Plus (17.2)](https://testingbot.com# "iPhone 15 Plus - 17.2") [iPhone 15 Pro (17.2)](https://testingbot.com# "iPhone 15 Pro - 17.2") [iPhone 15 Pro Max (17.2)](https://testingbot.com# "iPhone 15 Pro Max - 17.2") [iPhone SE (3rd generation) (17.2)](https://testingbot.com# "iPhone SE (3rd generation) - 17.2") #### 17.0 [iPhone 15 (17.0)](https://testingbot.com# "iPhone 15 - 17.0") [iPhone 15 Plus (17.0)](https://testingbot.com# "iPhone 15 Plus - 17.0") [iPhone 15 Pro (17.0)](https://testingbot.com# "iPhone 15 Pro - 17.0") [iPhone 15 Pro Max (17.0)](https://testingbot.com# "iPhone 15 Pro Max - 17.0") [iPhone SE (3rd generation) (17.0)](https://testingbot.com# "iPhone SE (3rd generation) - 17.0") #### 16.2 [iPhone 14 (16.2)](https://testingbot.com# "iPhone 14 - 16.2") [iPhone 14 Plus (16.2)](https://testingbot.com# "iPhone 14 Plus - 16.2") [iPhone 14 Pro (16.2)](https://testingbot.com# "iPhone 14 Pro - 16.2") [iPhone 14 Pro Max (16.2)](https://testingbot.com# "iPhone 14 Pro Max - 16.2") #### 16.0 [iPhone 14 (16.0)](https://testingbot.com# "iPhone 14 - 16.0") [iPhone 14 Pro (16.0)](https://testingbot.com# "iPhone 14 Pro - 16.0") [iPhone 14 Pro Max (16.0)](https://testingbot.com# "iPhone 14 Pro Max - 16.0") [iPhone 14 Plus (16.0)](https://testingbot.com# "iPhone 14 Plus - 16.0") [iPhone SE (3rd generation) (16.0)](https://testingbot.com# "iPhone SE (3rd generation) - 16.0") #### 15.4 [iPhone 13 (15.4)](https://testingbot.com# "iPhone 13 - 15.4") [iPhone 13 Pro (15.4)](https://testingbot.com# "iPhone 13 Pro - 15.4") [iPhone 13 Pro Max (15.4)](https://testingbot.com# "iPhone 13 Pro Max - 15.4") [iPhone 13 mini (15.4)](https://testingbot.com# "iPhone 13 mini - 15.4") [iPhone 12 Pro Max (15.4)](https://testingbot.com# "iPhone 12 Pro Max - 15.4") [iPhone 12 Pro (15.4)](https://testingbot.com# "iPhone 12 Pro - 15.4") [iPhone 12 (15.4)](https://testingbot.com# "iPhone 12 - 15.4") [iPhone 11 Pro Max (15.4)](https://testingbot.com# "iPhone 11 Pro Max - 15.4") [iPhone 11 Pro (15.4)](https://testingbot.com# "iPhone 11 Pro - 15.4") [iPhone 11 (15.4)](https://testingbot.com# "iPhone 11 - 15.4") [iPhone 8 Plus (15.4)](https://testingbot.com# "iPhone 8 Plus - 15.4") [iPhone 8 (15.4)](https://testingbot.com# "iPhone 8 - 15.4") #### 15.0 [iPhone 13 (15.0)](https://testingbot.com# "iPhone 13 - 15.0") [iPhone 13 Pro (15.0)](https://testingbot.com# "iPhone 13 Pro - 15.0") [iPhone 13 Pro Max (15.0)](https://testingbot.com# "iPhone 13 Pro Max - 15.0") [iPhone 12 Pro Max (15.0)](https://testingbot.com# "iPhone 12 Pro Max - 15.0") [iPhone 12 Pro (15.0)](https://testingbot.com# "iPhone 12 Pro - 15.0") [iPhone 12 (15.0)](https://testingbot.com# "iPhone 12 - 15.0") [iPhone 11 Pro Max (15.0)](https://testingbot.com# "iPhone 11 Pro Max - 15.0") [iPhone 11 Pro (15.0)](https://testingbot.com# "iPhone 11 Pro - 15.0") [iPhone 11 (15.0)](https://testingbot.com# "iPhone 11 - 15.0") [iPhone 8 Plus (15.0)](https://testingbot.com# "iPhone 8 Plus - 15.0") [iPhone 8 (15.0)](https://testingbot.com# "iPhone 8 - 15.0") #### Tablets #### 27.0 [iPad mini (A17 Pro) (27.0)](https://testingbot.com# "iPad mini (A17 Pro) - 27.0") [iPad (A16) (27.0)](https://testingbot.com# "iPad (A16) - 27.0") [iPad Pro 13-inch (M5) (27.0)](https://testingbot.com# "iPad Pro 13-inch (M5) - 27.0") [iPad Air 13-inch (M4) (27.0)](https://testingbot.com# "iPad Air 13-inch (M4) - 27.0") [iPad Pro 11-inch (M5) (27.0)](https://testingbot.com# "iPad Pro 11-inch (M5) - 27.0") [iPad Air 11-inch (M4) (27.0)](https://testingbot.com# "iPad Air 11-inch (M4) - 27.0") #### 26.3 [iPad mini (A17 Pro) (26.3)](https://testingbot.com# "iPad mini (A17 Pro) - 26.3") [iPad (A16) (26.3)](https://testingbot.com# "iPad (A16) - 26.3") [iPad Pro (13-inch) (M5) (26.3)](https://testingbot.com# "iPad Pro (13-inch) (M5) - 26.3") [iPad Air 13-inch (M3) (26.3)](https://testingbot.com# "iPad Air 13-inch (M3) - 26.3") [iPad Pro (11-inch) (M5) (26.3)](https://testingbot.com# "iPad Pro (11-inch) (M5) - 26.3") [iPad Air 11-inch (M3) (26.3)](https://testingbot.com# "iPad Air 11-inch (M3) - 26.3") #### 26.0 [iPad (A16) (26.0)](https://testingbot.com# "iPad (A16) - 26.0") [iPad Pro (13-inch) (M5) (26.0)](https://testingbot.com# "iPad Pro (13-inch) (M5) - 26.0") [iPad Air 13-inch (M3) (26.0)](https://testingbot.com# "iPad Air 13-inch (M3) - 26.0") [iPad Pro (11-inch) (M5) (26.0)](https://testingbot.com# "iPad Pro (11-inch) (M5) - 26.0") [iPad Air 11-inch (M3) (26.0)](https://testingbot.com# "iPad Air 11-inch (M3) - 26.0") #### 18.6 [iPad Mini (A17 Pro) (18.6)](https://testingbot.com# "iPad Mini (A17 Pro) - 18.6") [iPad (A16) (18.6)](https://testingbot.com# "iPad (A16) - 18.6") [iPad Pro (13-inch) (M4) (18.6)](https://testingbot.com# "iPad Pro (13-inch) (M4) - 18.6") [iPad Air 13-inch (M3) (18.6)](https://testingbot.com# "iPad Air 13-inch (M3) - 18.6") [iPad Pro (11-inch) (M4) (18.6)](https://testingbot.com# "iPad Pro (11-inch) (M4) - 18.6") [iPad Air 11-inch (M3) (18.6)](https://testingbot.com# "iPad Air 11-inch (M3) - 18.6") #### 18.3 [iPad (A16) (18.3)](https://testingbot.com# "iPad (A16) - 18.3") [iPad Pro (13-inch) (M4) (18.3)](https://testingbot.com# "iPad Pro (13-inch) (M4) - 18.3") [iPad Air 13-inch (M3) (18.3)](https://testingbot.com# "iPad Air 13-inch (M3) - 18.3") [iPad Air 11-inch (M3) (18.3)](https://testingbot.com# "iPad Air 11-inch (M3) - 18.3") [iPad Pro (11-inch) (M4) (18.3)](https://testingbot.com# "iPad Pro (11-inch) (M4) - 18.3") #### 18.0 [iPad Pro (13-inch) (M4) (18.0)](https://testingbot.com# "iPad Pro (13-inch) (M4) - 18.0") [iPad Air 13-inch (M2) (18.0)](https://testingbot.com# "iPad Air 13-inch (M2) - 18.0") [iPad Pro (11-inch) (M4) (18.0)](https://testingbot.com# "iPad Pro (11-inch) (M4) - 18.0") [iPad Air 11-inch (M2) (18.0)](https://testingbot.com# "iPad Air 11-inch (M2) - 18.0") #### 17.5 [iPad Air 13-inch (M2) (17.5)](https://testingbot.com# "iPad Air 13-inch (M2) - 17.5") [iPad Pro (13-inch) (M4) (17.5)](https://testingbot.com# "iPad Pro (13-inch) (M4) - 17.5") [iPad Air 11-inch (M2) (17.5)](https://testingbot.com# "iPad Air 11-inch (M2) - 17.5") [iPad Pro (11-inch) (M4) (17.5)](https://testingbot.com# "iPad Pro (11-inch) (M4) - 17.5") #### 17.4 [iPad Pro (13-inch) (M4) (17.4)](https://testingbot.com# "iPad Pro (13-inch) (M4) - 17.4") [iPad Air 13-inch (M2) (17.4)](https://testingbot.com# "iPad Air 13-inch (M2) - 17.4") [iPad Pro (11-inch) (M4) (17.4)](https://testingbot.com# "iPad Pro (11-inch) (M4) - 17.4") [iPad Air 11-inch (M2) (17.4)](https://testingbot.com# "iPad Air 11-inch (M2) - 17.4") #### 17.2 [iPad Pro (12.9-inch) (6th generation) (17.2)](https://testingbot.com# "iPad Pro (12.9-inch) (6th generation) - 17.2") [iPad Pro (11-inch) (4th generation) (17.2)](https://testingbot.com# "iPad Pro (11-inch) (4th generation) - 17.2") [iPad (10th generation) (17.2)](https://testingbot.com# "iPad (10th generation) - 17.2") [iPad mini (6th generation) (17.2)](https://testingbot.com# "iPad mini (6th generation) - 17.2") [iPad Air (5th generation) (17.2)](https://testingbot.com# "iPad Air (5th generation) - 17.2") #### 17.0 [iPad Pro (12.9-inch) (6th generation) (17.0)](https://testingbot.com# "iPad Pro (12.9-inch) (6th generation) - 17.0") [iPad Pro (11-inch) (4th generation) (17.0)](https://testingbot.com# "iPad Pro (11-inch) (4th generation) - 17.0") [iPad (10th generation) (17.0)](https://testingbot.com# "iPad (10th generation) - 17.0") [iPad mini (6th generation) (17.0)](https://testingbot.com# "iPad mini (6th generation) - 17.0") [iPad Air (5th generation) (17.0)](https://testingbot.com# "iPad Air (5th generation) - 17.0") #### 16.2 [iPad Pro (12.9-inch) (6th generation) (16.2)](https://testingbot.com# "iPad Pro (12.9-inch) (6th generation) - 16.2") [iPad Pro (11-inch) (4th generation) (16.2)](https://testingbot.com# "iPad Pro (11-inch) (4th generation) - 16.2") [iPad (10th generation) (16.2)](https://testingbot.com# "iPad (10th generation) - 16.2") [iPad Air (5th generation) (16.2)](https://testingbot.com# "iPad Air (5th generation) - 16.2") #### 16.0 [iPad Pro (12.9-inch) (5th generation) (16.0)](https://testingbot.com# "iPad Pro (12.9-inch) (5th generation) - 16.0") [iPad Pro (11-inch) (3rd generation) (16.0)](https://testingbot.com# "iPad Pro (11-inch) (3rd generation) - 16.0") [iPad (9th generation) (16.0)](https://testingbot.com# "iPad (9th generation) - 16.0") [iPad Air (5th generation) (16.0)](https://testingbot.com# "iPad Air (5th generation) - 16.0") #### 15.4 [iPad Pro (12.9-inch) (5th generation) (15.4)](https://testingbot.com# "iPad Pro (12.9-inch) (5th generation) - 15.4") [iPad Pro (11-inch) (3rd generation) (15.4)](https://testingbot.com# "iPad Pro (11-inch) (3rd generation) - 15.4") [iPad (9th generation) (15.4)](https://testingbot.com# "iPad (9th generation) - 15.4") [iPad mini (6th generation) (15.4)](https://testingbot.com# "iPad mini (6th generation) - 15.4") [iPad Air (5th generation) (15.4)](https://testingbot.com# "iPad Air (5th generation) - 15.4") #### 15.0 [iPad Pro (12.9-inch) (5th generation) (15.0)](https://testingbot.com# "iPad Pro (12.9-inch) (5th generation) - 15.0") [iPad Pro (11-inch) (3rd generation) (15.0)](https://testingbot.com# "iPad Pro (11-inch) (3rd generation) - 15.0") [iPad (9th generation) (15.0)](https://testingbot.com# "iPad (9th generation) - 15.0") [iPad mini (6th generation) (15.0)](https://testingbot.com# "iPad mini (6th generation) - 15.0") [iPad Air (4th generation) (15.0)](https://testingbot.com# "iPad Air (4th generation) - 15.0") ![Android Browser Testing](https://testingbot.com/assets/environments/svg/android-1b3d235e17522e2238e1c55b4e9cdd41219f653cd958ea065877d3813f32ffc3.svg) ## Android Emulators ### Phones #### 16.0 [Pixel 9 (16.0)](https://testingbot.com# "Pixel 9 - 16.0") #### 15.0 [Pixel 9 (15.0)](https://testingbot.com# "Pixel 9 - 15.0") #### 14.0 [Pixel 8 (14.0)](https://testingbot.com# "Pixel 8 - 14.0") #### 13.0 [Pixel 6a (13.0)](https://testingbot.com# "Pixel 6a - 13.0") #### 12.0 [Pixel 6 (12.0)](https://testingbot.com# "Pixel 6 - 12.0") #### 11.0 [Galaxy S21 (11.0)](https://testingbot.com# "Galaxy S21 - 11.0") #### 10.0 [Pixel 3 (10.0)](https://testingbot.com# "Pixel 3 - 10.0") [Galaxy S10 (10.0)](https://testingbot.com# "Galaxy S10 - 10.0") #### 9.0 [Galaxy S9 (9.0)](https://testingbot.com# "Galaxy S9 - 9.0") #### 8.0 [Pixel 2 XL (8.0)](https://testingbot.com# "Pixel 2 XL - 8.0") #### 7.1 [Pixel 2 (7.1)](https://testingbot.com# "Pixel 2 - 7.1") [Nexus 7 (7.1)](https://testingbot.com# "Nexus 7 - 7.1") #### 6.0 [Nexus S (6.0)](https://testingbot.com# "Nexus S - 6.0") [Nexus 6 (6.0)](https://testingbot.com# "Nexus 6 - 6.0") [Galaxy S6 (6.0)](https://testingbot.com# "Galaxy S6 - 6.0") [Nexus 4 (6.0)](https://testingbot.com# "Nexus 4 - 6.0") #### 5.0 [Galaxy S5 (5.0)](https://testingbot.com# "Galaxy S5 - 5.0") [Nexus 1 (5.0)](https://testingbot.com# "Nexus 1 - 5.0") #### 4.4 [Galaxy S4 (4.4)](https://testingbot.com# "Galaxy S4 - 4.4") [Galaxy Nexus (4.4)](https://testingbot.com# "Galaxy Nexus - 4.4") ### Tablets #### 15.0 [Galaxy Tab S9 (15.0)](https://testingbot.com# "Galaxy Tab S9 - 15.0") #### 12.0 [Galaxy Tab S7 (12.0)](https://testingbot.com# "Galaxy Tab S7 - 12.0") #### 11.0 [Galaxy Tab S6 (11.0)](https://testingbot.com# "Galaxy Tab S6 - 11.0") #### 10.0 [Galaxy Tab S5 (10.0)](https://testingbot.com# "Galaxy Tab S5 - 10.0") #### 9.0 [Galaxy Tab S4 (9.0)](https://testingbot.com# "Galaxy Tab S4 - 9.0") #### 8.0 [Pixel C (8.0)](https://testingbot.com# "Pixel C - 8.0") ### Chromeos #### 14.0 [ChromeOS Medium (14.0)](https://testingbot.com# "ChromeOS Medium - 14.0") [ChromeOS Large (14.0)](https://testingbot.com# "ChromeOS Large - 14.0") [ChromeOS Small (14.0)](https://testingbot.com# "ChromeOS Small - 14.0") --- URL: https://testingbot.com/support/web-automate/vibium # Vibium Testing [Vibium](https://vibium.com) is an open-source browser automation tool built for AI (coding) agents. It comes with a CLI and an MCP server, so both you and your AI agent can drive a real browser: navigate to pages, take screenshots, extract text and verify that generated code actually works. Vibium connects to TestingBot through the regular WebDriver endpoint at `hub.testingbot.com/wd/hub`. Vibium creates the session with `webSocketUrl: true` and attaches to the WebDriver BiDi socket that the TestingBot grid returns, so no extra configuration is required. ## Installation Install the Vibium CLI globally with npm: ```console npm install -g vibium ``` Alternatively, you can run Vibium without installing it, via `npx -y vibium`. ## Connect to TestingBot Point `vibium start` to the TestingBot hub. Your TestingBot `key` and `secret` (obtained from the [member dashboard](https://testingbot.com/members/user/edit)) go in the username and password slots of the URL: ```bash export TESTINGBOT_KEY="your_key" export TESTINGBOT_SECRET="your_secret" # Optional: capabilities for the session, "name" labels it in the TestingBot dashboard export VIBIUM_CONNECT_CAPS='{"browserName":"chrome","tb:options":{"name":"vibium"}}' vibium start "https://$TESTINGBOT_KEY:$TESTINGBOT_SECRET@hub.testingbot.com/wd/hub" ``` Once the session is started, every Vibium command runs against the cloud browser: ```bash # Navigate and read the page vibium go https://duckduckgo.com vibium title # Find elements by their semantic attributes; this returns a stable reference such as @e1 vibium find placeholder "Search privately" # Fill the input and submit vibium fill @e1 "TestingBot" vibium press Enter # Wait for the results, then capture them vibium wait text "TestingBot" vibium screenshot -o results.png vibium text > results.txt ``` The test, together with a video recording and logs, will appear in the [TestingBot dashboard](https://testingbot.com/members) while it runs. ## MCP Server Vibium also ships an MCP server, which allows AI agents such as Claude Code to control the browser through MCP tools. It reads the same connection settings from the environment; with `VIBIUM_CONNECT_CAPS` still exported from above: ```bash VIBIUM_CONNECT_URL="https://$TESTINGBOT_KEY:$TESTINGBOT_SECRET@hub.testingbot.com/wd/hub" vibium mcp ``` Your AI agent will now open its browser sessions on TestingBot instead of on your local machine. ## Client Libraries Next to the CLI, Vibium provides client libraries for JavaScript/TypeScript, Python and Java. Pass the TestingBot hub URL when starting the browser session, or export `VIBIUM_CONNECT_URL` in your environment. The `VIBIUM_CONNECT_CAPS` environment variable from above works for the client libraries as well. [JavaScript](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) ```bash npm install vibium ``` ```javascript import { writeFileSync } from 'node:fs' import { browser } from 'vibium' const hub = `https://${process.env.TESTINGBOT_KEY}:${process.env.TESTINGBOT_SECRET}@hub.testingbot.com/wd/hub` const browserSession = await browser.start(hub) const vibe = await browserSession.page() await vibe.go('https://testingbot.com') console.log(await vibe.title()) writeFileSync('example.png', await vibe.screenshot()) await browserSession.stop() ``` ```bash pip install vibium ``` ```python import os from vibium import browser hub = f"https://{os.environ['TESTINGBOT_KEY']}:{os.environ['TESTINGBOT_SECRET']}@hub.testingbot.com/wd/hub" browser_session = browser.start(hub) vibe = browser_session.page() vibe.go("https://testingbot.com") print(vibe.title()) browser_session.stop() ``` The Java client picks up the connection from the `VIBIUM_CONNECT_URL` environment variable: ```bash export VIBIUM_CONNECT_URL="https://$TESTINGBOT_KEY:$TESTINGBOT_SECRET@hub.testingbot.com/wd/hub" ``` ```java var browserSession = Vibium.start(); var vibe = browserSession.page(); vibe.go("https://testingbot.com"); System.out.println(vibe.title()); var png = vibe.screenshot(); java.nio.file.Files.write(java.nio.file.Path.of("example.png"), png); browserSession.stop(); ``` ## Specify Browsers & Devices The `VIBIUM_CONNECT_CAPS` environment variable holds the WebDriver capabilities for the session. Use it to pick a browser, browser version and operating system, and to set TestingBot specific options in `tb:options`. To see how to do this, please select a combination of browser, version and platform in the drop-down menus below. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. See the [Test Configuration Options](https://testingbot.com/support/web-automate/selenium/test-options) page for all the options that `tb:options` supports, such as video recording, timeouts and geolocation. ## Stop the Session When you are done, stop Vibium to delete the WebDriver session. This immediately releases your parallel test slot: ```console vibium stop ``` ## Testing Internal Websites The browsers in the TestingBot cloud can only reach public URLs. To test a website on `localhost` or inside a private network, use the [TestingBot Tunnel](https://testingbot.com/support/tunnel), which creates a secure connection between your machine and the TestingBot browser grid. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium Documentation # Selenium Testing in the Cloud TestingBot is a Selenium testing cloud that runs your WebDriver tests on 6100+ real desktop browsers and physical iOS and Android devices without managing local infrastructure. Point your existing Selenium scripts at our remote grid endpoint and run tests in parallel across Chrome, Firefox, Safari, Edge and Internet Explorer. Test across Windows, macOS, Linux, iOS and Android. ## Quick start: run your first Selenium test Four steps to point an existing Selenium WebDriver test at the TestingBot cloud: 1. **Sign up and grab your API key.** [Create a free account](https://testingbot.com/users/sign_up). Your API key and secret are in the dashboard under _Account → Keys & Authentication_. 2. **Point Selenium at the TestingBot grid.** Change your WebDriver endpoint to `https://hub.testingbot.com/wd/hub`. Pass your key and secret via the `tb:options` capability. 3. **Set desired capabilities.** Pick a browser, version and platform, for example `browserName: "chrome"`, `browserVersion: "latest"`, `platformName: "WIN11"`. Add an optional `name` and `build` for grouping in the dashboard. 4. **Run your test and view results.** TestingBot streams a live video, screenshots, browser logs and Selenium command logs into the dashboard while the test runs and after it finishes. ## Pick your language or framework Each card below links to a complete, runnable example for that language or framework, including [desired capabilities](https://testingbot.com/support/web-automate/selenium/test-options), common patterns and CI examples. ### [NodeJS](https://testingbot.com/support/web-automate/selenium/nodejs) Get started with NodeJS testing frameworks and tools. - [NodeJS](https://testingbot.com/support/web-automate/selenium/nodejs) - [WebdriverIO](https://testingbot.com/support/web-automate/selenium/nodejs/webdriverio) - [CodeceptJS](https://testingbot.com/support/web-automate/selenium/nodejs/codeceptjs) - [Nightwatch](https://testingbot.com/support/web-automate/selenium/nodejs/nightwatch) - [Protractor](https://testingbot.com/support/web-automate/selenium/nodejs/protractor) - [Soda](https://testingbot.com/support/web-automate/selenium/nodejs/soda) - [TestCafe](https://testingbot.com/support/web-automate/selenium/nodejs/testcafe) - [Hermione](https://testingbot.com/support/web-automate/selenium/nodejs/hermione) - [Jest](https://testingbot.com/support/web-automate/selenium/nodejs/jest) ### [Java](https://testingbot.com/support/web-automate/selenium/java) Get started with Java testing frameworks and tools. - [Java](https://testingbot.com/support/web-automate/selenium/java) - [JUnit](https://testingbot.com/support/web-automate/selenium/java/junit) - [Parallel JUnit](https://testingbot.com/support/web-automate/selenium/java/parallel-junit) - [Selenide](https://testingbot.com/support/web-automate/selenium/java/selenide) - [TestNG](https://testingbot.com/support/web-automate/selenium/java/testng) - [TestNG + Cucumber](https://testingbot.com/support/web-automate/selenium/java/testng-cucumber) ### [Python](https://testingbot.com/support/web-automate/selenium/python) Get started with Python testing frameworks and tools. - [Python](https://testingbot.com/support/web-automate/selenium/python) - [PyTest](https://testingbot.com/support/web-automate/selenium/python/pytest) - [PyUnit](https://testingbot.com/support/web-automate/selenium/python/pyunit) - [Behave](https://testingbot.com/support/web-automate/selenium/python/behave) - [Lettuce](https://testingbot.com/support/web-automate/selenium/python/lettuce) - [Helium](https://testingbot.com/support/web-automate/selenium/python/helium) - [SeleniumBase](https://testingbot.com/support/web-automate/selenium/python/seleniumbase) ### [.NET](https://testingbot.com/support/web-automate/selenium/csharp) Get started with .NET testing frameworks and tools. - [C#](https://testingbot.com/support/web-automate/selenium/csharp) - [NUnit](https://testingbot.com/support/web-automate/selenium/csharp/nunit) - [Reqnroll](https://testingbot.com/support/web-automate/selenium/csharp/reqnroll) - [xUnit](https://testingbot.com/support/web-automate/selenium/csharp/xunit) - [PNunit](https://testingbot.com/support/web-automate/selenium/csharp/pnunit) - [SpecFlow](https://testingbot.com/support/web-automate/selenium/csharp/specflow) - [MSTest](https://testingbot.com/support/web-automate/selenium/csharp/mstest) - [MbUnit](https://testingbot.com/support/web-automate/selenium/csharp/mbunit) ### [PHP](https://testingbot.com/support/web-automate/selenium/php) Get started with PHP testing frameworks and tools. - [PHP](https://testingbot.com/support/web-automate/selenium/php) - [Behat](https://testingbot.com/support/web-automate/selenium/php/behat-mink) - [Codeception](https://testingbot.com/support/web-automate/selenium/php/codeception) - [Laravel-Dusk](https://testingbot.com/support/web-automate/selenium/php/laravel-dusk) - [PHPUnit](https://testingbot.com/support/web-automate/selenium/php/phpunit) - [SimpleTest](https://testingbot.com/support/web-automate/selenium/php/simpletest) ### [Ruby](https://testingbot.com/support/web-automate/selenium/ruby) Get started with Ruby testing frameworks and tools. - [Ruby](https://testingbot.com/support/web-automate/selenium/ruby) - [Capybara](https://testingbot.com/support/web-automate/selenium/ruby/capybara) - [Cucumber](https://testingbot.com/support/web-automate/selenium/ruby/cucumber) - [Minitest](https://testingbot.com/support/web-automate/selenium/ruby/minitest) - [RSpec](https://testingbot.com/support/web-automate/selenium/ruby/rspec) - [Test::Unit](https://testingbot.com/support/web-automate/selenium/ruby/testunit) - [Watir](https://testingbot.com/support/web-automate/selenium/ruby/watir) ### [JavaScript](https://testingbot.com/support/web-automate/selenium/javascript) Get started with JavaScript testing frameworks and tools. - [JavaScript](https://testingbot.com/support/web-automate/selenium/javascript) - [Intern](https://testingbot.com/support/web-automate/selenium/javascript/intern) - [Karma](https://testingbot.com/support/web-automate/selenium/javascript/karma) ### [Appium](https://testingbot.com/support/web-automate/mobile) Get started with mobile web testing using Appium. - [Appium Examples](https://testingbot.com/support/web-automate/mobile) ![Robot Framework Testing example](https://testingbot.com/assets/wizard/robotframework-08821f4942343bcb1f828470b5cbff3163369c14884b7ca285be190017ee7a94.png) ### [Robot Framework](https://testingbot.com/support/web-automate/selenium/robotframework) Get started with Robot Framework test automation. - [RobotFramework Examples](https://testingbot.com/support/web-automate/selenium/robotframework) ### Other Get started with other testing tools and IDEs. - [Selenium IDE](https://testingbot.com/support/web-automate/selenium/ide) - [Katalon Studio](https://testingbot.com/support/web-automate/selenium/katalon-studio) - [Oxygen](https://testingbot.com/support/web-automate/selenium/oxygen) ## Selenium vs Playwright vs Cypress for cloud testing Selenium is the open-source standard for browser automation across the widest range of browsers and languages. Playwright and Cypress are newer frameworks with different trade-offs. All three run on TestingBot, pick the one that fits your stack: | | Selenium | Playwright | Cypress | | Browsers | Chrome, Firefox, Safari, Edge, Internet Explorer | Chromium, Firefox, WebKit | Chrome, Firefox, Edge, Electron | | Real mobile devices | Yes (Chrome on Android, Safari on iOS) | Emulated viewport only | Emulated viewport only | | Languages | Java, Python, C#, Ruby, JavaScript, PHP, Kotlin | JavaScript, Python, Java, .NET | JavaScript, TypeScript | | Parallel execution | Native via the grid; concurrency by plan | Native worker-based parallelism | Parallel via cloud or Cypress Cloud | | Protocol | W3C WebDriver (standard) | Direct CDP / WebKit protocols | Runs inside the browser as a Node process | | Best for | Broad browser coverage, multi-language teams, real-device testing | Modern web apps, fast Chromium-first feedback | JS / TS teams wanting an integrated runner | See dedicated guides for [Playwright on TestingBot](https://testingbot.com/support/web-automate/playwright) and [Cypress on TestingBot](https://testingbot.com/support/web-automate/cypress), or compare in depth on [Playwright vs Selenium](https://testingbot.com/playwright-vs-selenium). ## Frequently asked questions What is a Selenium testing cloud? A Selenium testing cloud is a hosted grid of remote browsers and devices that your Selenium WebDriver scripts connect to instead of running locally. TestingBot maintains 6100+ browser, operating system and version combinations including real iOS and Android devices, so you do not have to install drivers, browsers, or maintain a Selenium Grid yourself. How do I run Selenium tests in the cloud with TestingBot? Sign up for a free account, copy your API key and secret from the dashboard, and change your Selenium WebDriver endpoint to `https://hub.testingbot.com/wd/hub`. Add desired capabilities (browserName, browserVersion, platformName) and run your existing tests. No code changes are required beyond the URL and credentials. Which browsers and versions does TestingBot support? TestingBot supports Chrome, Firefox, Safari, Microsoft Edge and Internet Explorer on Windows, macOS and Linux, plus Chrome and Safari on real iOS and Android phones and tablets. See the [full browser list](https://testingbot.com/support/web-automate/browsers) for current versions. Can I run Selenium tests in parallel? Yes. TestingBot supports parallel test execution across multiple browsers and devices simultaneously. The number of concurrent tests is determined by your plan. Tests run on isolated, freshly-provisioned VMs so they never interfere with each other. Do I need to change my Selenium tests to run on TestingBot? No. Any test written against the standard Selenium WebDriver protocol works as-is on TestingBot. The only required changes are the remote URL (https://hub.testingbot.com/wd/hub) and your API key and secret. Your assertions, page objects and frameworks (JUnit, pytest, RSpec, Mocha, Cucumber and others) stay the same. How is TestingBot different from running my own Selenium Grid? With TestingBot you skip the infrastructure: no nodes to provision, no browsers or drivers to install, no version mismatches when a browser auto-updates, no flaky cleanup between tests. Every test runs on a fresh VM that is destroyed afterwards. Screenshots, video recording, network logs and a dashboard are included out of the box. Does TestingBot support real mobile devices for Selenium testing? Yes. In addition to desktop browsers, TestingBot runs Selenium tests against Chrome on real Android devices and Safari on real iOS devices housed in our EU data centre. See [real device testing](https://testingbot.com/real-device-testing) for details. How much does Selenium cloud testing cost? TestingBot offers a free trial with included minutes, plus paid plans starting from $30 per month. Pricing is based on test concurrency and total minutes. See [pricing](https://testingbot.com/pricing). Approved open-source projects can apply for a free plan. ## Next steps - [ **Java tutorial** Selenium WebDriver examples in Java](https://testingbot.com/support/web-automate/selenium/java) - [ **Python tutorial** Selenium WebDriver examples in Python](https://testingbot.com/support/web-automate/selenium/python) - [ **Node.js tutorial** Selenium WebDriver examples in Node.js](https://testingbot.com/support/web-automate/selenium/nodejs) - [ **Capabilities reference** All tb:options capabilities](https://testingbot.com/support/web-automate/selenium/test-options) - [ **Browser list** All 6100+ supported browser combinations](https://testingbot.com/support/web-automate/browsers) - [ **Real device testing** Selenium on real iOS and Android](https://testingbot.com/real-device-testing) - [ **Selenium IDE** Record and replay without writing code](https://testingbot.com/support/web-automate/selenium/ide) --- URL: https://testingbot.com/support/web-automate/selenium/live-view # Test Live View When running automated Selenium tests with TestingBot, you can see a realtime view of the test while it is running. If you are running the tests in a CI/CD pipeline, or on your own machines, it might be useful to see what is happening during the test execution. TestingBot offers a realtime live view of your Selenium tests, so you can see what is happening during the test execution. You can embed this live view in your own dashboards, or open it in a separate window. The live view also supports human in the loop: you can take control of the browser during the test execution, with mouse and keyboard. Realtime Views can be useful for: - Debugging tests by seeing what is happening during the test execution or share a view/control link with colleagues. - Human in the loop: instantly take control or provide input during a test. For example if the test gets stuck waiting for a Captcha or user input (for example authentication). - Embedding in other applications or dashboards for a unified view of test progress. ## Getting started You will need to generate a specific URL to access the live view of your test. To generate the URL, you will need the [WebDriver sessionId](https://testingbot.com/support/web-automate/selenium/get-session-id) of the test that is running. Once you have the sessionId, you can generate an authentication token. The authentication hash is created with MD5 in the following format: [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) ```java TestingbotREST restApi = new TestingbotREST("key", "secret"); String hash = restApi.getAuthenticationHash(sessionId); ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient("key", "secret") tb.get_share_link(session_id) ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const hash = api.getAuthenticationHashForSharing(sessionId); ``` ```ruby require 'testingbot' api = TestingBot::Api.new("key", "secret") api.get_authentication_hash(session_id) ``` ```php $api = new TestingBot\TestingBotAPI("key", "secret"); $api->getAuthenticationHash($sessionId); ``` Now that you have the authentication hash, you can construct the live view URL as follows: ``` https://testingbot.com/tests//live?auth= ``` ## Embed Once you've constructed the live view URL, you can embed it in an iframe within your own application or dashboard: ```html ``` Adjust the width and height attributes as needed to fit your layout. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/annotating-tests # Updating Tests with Selenium You can use the TestingBot API to send various test information back to TestingBot: test-name, passed/failed state, build identifier and more. We've added custom [JavaScript Executor](https://www.selenium.dev/selenium/docs/api/java/org/openqa/selenium/JavascriptExecutor.html) commands which allow you to send back test meta-data to TestingBot, while your test is running. Below are some examples on how to do this in various programming languages: [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java ((JavascriptExecutor)driver).executeScript("tb:test-name=My test"); ((JavascriptExecutor)driver).executeScript("tb:test-result=passed"); ``` ```python driver.execute_script('tb:test-name=My test') driver.execute_script('tb:test-result=passed') ``` ```javascript await driver.executeScript('tb:test-name=My test'); await driver.executeScript('tb:test-result=passed'); ``` ```ruby driver.execute_script('tb:test-name=My test') driver.execute_script('tb:test-result=passed') ``` ```php $web_driver->executeScript('tb:test-name=My test'); $web_driver->executeScript('tb:test-result=passed'); ``` ```csharp ((IJavaScriptExecutor)driver).ExecuteScript("tb:test-name=My test"); ((IJavaScriptExecutor)driver).ExecuteScript("tb:test-result=passed"); ``` ## Set a test name ```javascript tb:test-name=My Test Name ``` Sets the test-name for the currently running test. This name will be visible on TestingBot. ## Break test ```javascript tb:break ``` This will add a breakpoint to the test and pause it, so that you can investigate it manually. When you go to the test result page on TestingBot, a live session will appear so that you can debug with mouse and keyboard. ## Test Result ```javascript tb:test-result=true ``` Possible values: passed, true, failed, false This will set the test as passed (`"passed"` or `true`) or failed (`"failed"` or `false`) on TestingBot. ## Test Build ```javascript tb:test-build=my build ``` Sets the test's build name. ## Test Tags ```javascript tb:test-tags=tag1,tag2 ``` Comma-separated list of tags. Sets the test's tags. ## Test Context ```javascript tb:test-context=my context ``` Logs the given context and prints it on the TestingBot test result page, in the Selenium command list. ## Test Info ```javascript tb:test-info={'build':'my first build', 'name':'my new test', 'public':true} ``` JSON-formatted dictionary with optional keys: `'build'`, `'name'`, `'public'`, `'status_message'`, `'extra'` Pass in a JSON dictionary with various fields to update the test's meta-data on TestingBot. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/capabilities # Selenium Capabilities Generator Generate Selenium capabilities for your automated tests. Choose your browser, OS and configuration options to get the exact code you need for Java, Python, NodeJS, C#, Ruby or PHP. [![Selenium capabilities](https://testingbot.com/assets/support/selenium-266ecc0599068fc6eca0bcd9174fb46053c95cebe1c66f7a50c480f3f1a52929.svg) Selenium ](https://testingbot.com/support/web-automate/selenium/capabilities) [![Appium capabilities](https://testingbot.com/assets/support/appium-4b35a8bb56bae236bd84c539cee9b8bbb489a55ddf48e32d059c86685d230354.svg) Appium ](https://testingbot.com/support/app-automate/appium/capabilities) [![Puppeteer capabilities](https://testingbot.com/assets/support/puppeteer-0a8cdd38af69dabd46638afbb3bffc8a372902949aa599663ec379f3e875263c.svg) Puppeteer ](https://testingbot.com/support/web-automate/puppeteer/capabilities) [![Playwright capabilities](https://testingbot.com/assets/support/playwright-2dcb4e75b55ac0d1135cfa47be6137b13f46a2047ce78b96a34f51b0e2cc0d96.svg) Playwright ](https://testingbot.com/support/web-automate/playwright/capabilities) Browser Environment ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. Resolution 800x600 1024x768 1280x800 1280x960 1280x1024 1440x900 1680x1050 1600x1200 1920x1200 1920x1080 2560x1440 Selenium Version 4.48.0 4.47.0 4.46.0 4.45.0 4.44.0 4.43.0 4.42.0 4.41.0 4.40.0 4.39.0 4.38.0 4.37.0 4.36.0 4.35.0 4.34.0 4.33.0 4.32.0 4.31.0 4.30.0 4.29.0 4.28.1 4.28.0 4.27.0 4.26.0 4.25.0 4.24.0 4.23.1 4.23.0 4.22.0 4.21.0 4.20.0 4.19.0 4.18.1 4.18.0 4.17.0 4.16.1 4.16.0 4.15.0 4.14.1 4.14.0 4.13.0 4.12.1 4.12.0 4.11.0 4.10.0 4.9.0 4.8.3 4.8.2 4.8.1 4.8.0 4.7.0 4.6.0 4.5.3 4.5.2 4.5.1 4.5.0 4.4.0 4.3.0 4.2.2 4.2.1 4.2.0 4.1.3 4.1.2 4.1.1 4.1.0 4.0.0 4.0.0-beta-4 4.0.0-beta-3 4.0.0-beta-2 4.0.0-beta-1 4.0.0-alpha-7 4.0.0-alpha-6 3.141.59 3.141.5 3.141.0 3.14.0 3.13.0 3.12.0 3.11.0 3.10.0 3.9.0 3.8.1 3.8.0 3.7.1 3.7.0 3.6.0 3.5.3 3.5.2 3.5.1 3.5.0 3.4.0 3.3.1 3.3.0 3.2.0 3.1.0 3.0.1 2.53.1 2.53.0 2.52.0 2.51.0 2.50.0 2.49.0 2.48.2 2.48.1 2.48.0 ## Project Capabilities Test Name Allows you to customize the name of the test, as it will appear in the dashboard. Build Name Group tests under the same build identifier for easy retrieval. ## Debugging Record Video Yes No Take Screenshot After Each Step Yes No Collect Logs Yes No ## Customization Headless Mode Run the browser without rendering UI for faster test execution. Yes No Geolocation Testing Run the test from a different geographical location. None Random Country Australia Belgium Brazil Canada Chile France Germany India Italy Japan Norway Singapore South Africa Sweden Switzerland United Arab Emirates United Kingdom United States Timezone ([list of timezones](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones)) Prerun Script ([documentation](https://testingbot.com/support/web-automate/selenium/test-options#prerun)) Upload URLs Looking for more options? See [all Selenium capabilities](https://testingbot.com/support/web-automate/selenium/test-options). W3C Protocol W3C is the newest WebDriver protocol, TestingBot recommends using W3C. JSONWP JSONWP is the legacy protocol which is no longer actively maintained. Java Python NodeJS C# Ruby PHP ```java MutableCapabilities capabilities = new MutableCapabilities(); capabilities.setCapability(CapabilityType.BROWSER_NAME, "chrome"); capabilities.setCapability(CapabilityType.BROWSER_VERSION, "latest"); capabilities.setCapability(CapabilityType.PLATFORM_NAME, "Windows 11"); ``` --- URL: https://testingbot.com/support/web-automate/selenium/performance # Performance Testing Documentation With this call, you can fetch various performance metrics from the browser, during your test. With this functionality, you can create tests where you monitor specific pageload time, or any other metric. If the metric is below a specific threshold that you define, you can make the test fail. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java ((JavascriptExecutor) driver).executeScript("tb:performance"); ``` ```python driver.execute_script("tb:performance") ``` ```javascript await driver.executeScript('tb:performance'); ``` ```ruby driver.execute_script("tb:performance") ``` ```php $driver->executeScript("tb:performance"); ``` ```csharp ((IJavaScriptExecutor)driver).ExecuteScript("tb:performance"); ``` ## Response Metrics | `pageSize (bytes)` | The size (in bytes) of the page (HTML source). | | --- | --- | | `nrRequests` | How many requests the page made (AJAX/images/...) | | `load (ms)` | The pageload time of the webpage. The time it takes for a page to download and display the entire web page. | | `domContentLoaded (ms)` | The domContentLoaded duration of the page. The DOMContentLoaded event fires when the initial HTML document has been completely loaded and parsed, without waiting for stylesheets, images, and subframes to finish loading. [DOMContentLoaded Info](https://developer.mozilla.org/en-US/docs/Web/Events/DOMContentLoaded) | | `firstMeaningfulPaint (ms)` | Indicates how fast your page displays its primary content. [firstMeaningfulPaint Info](https://developers.google.com/web/tools/lighthouse/audits/first-meaningful-paint) | | `firstPaint (ms)` | First Paint is triggered when a render (any render) is detected in the browser. | | `firstContentfulPaint (ms)` | First Contentful Paint measures the time from navigation to the time when the browser renders the first bit of content from the DOM. [firstContentfulPaint Info](https://developers.google.com/web/tools/lighthouse/audits/first-contentful-paint) | ### Example Response: ```json { "pageSize": 396800, "nrRequests": 20, "load": 1378, "domContentLoaded": 585, "firstMeaningfulPaint": 0, "firstPaint": 348, "firstContentfulPaint": 348 } ``` ## Sample Use Case This example demonstrates a test that visits a page, checks the performance and fails the test if the performance is not sufficient. ```ruby require "selenium-webdriver" options = Selenium::WebDriver::Chrome::Options.new options.add_option("platformName", "WIN11") options.add_option("browserVersion", "latest") options.add_option("tb:options", { key: "API_KEY", secret: "API_SECRET", name: "Performance Test" }) driver = Selenium::WebDriver.for :remote, url: "https://hub.testingbot.com/wd/hub", options: options driver.navigate.to "https://testingbot.com/" performance = driver.execute_script("tb:performance") if performance["load"] > 900 puts "Performance test failed: load time #{performance["load"]}ms exceeds 900ms threshold" end driver.quit ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/uploads # Uploading with Selenium With Selenium WebDriver it's possible to test file uploads on remote browsers and devices. There are several ways to handle file uploads depending on your use case: - [Upload a file before your test starts](https://testingbot.com#before), pre-load a file onto the test VM via a URL - [Upload a file during your test](https://testingbot.com#during), send a local file to the remote VM with `LocalFileDetector` - [Upload with WebdriverIO](https://testingbot.com#webdriverio), file upload examples for the WebdriverIO framework - [TestingBot Storage](https://testingbot.com#storage), use TestingBot's internal storage for faster file transfers - [Tips & Best Practices](https://testingbot.com#tips), common pitfalls and recommendations ## Which method should I use? - **Use [Upload Before Test](https://testingbot.com#before)** if you have a fixed test file at a public URL and want it ready on the VM before your test runs. - **Use [Upload During Test](https://testingbot.com#during)** if your test needs to interact with a file upload UI and send a file from your local machine to the remote browser. - **Use [TestingBot Storage](https://testingbot.com#storage)** if you have large files, private files, or need fast and consistent transfers across multiple test runs. ## 1. Upload a file before your test starts With TestingBot you can specify a URL in your capabilities which links to the file you want to be present on our test VMs. TestingBot will download the file and save it on the test VM before your test begins, so it's ready to use immediately. Use the following `tb:options` capabilities: - **`upload`:** The URL of the file to download onto the test VM. This can be a public URL or a `tb://` URL from [TestingBot Storage](https://testingbot.com#storage). - **`uploadFilepath`:** The absolute file path on the test VM where TestingBot should save the downloaded file (e.g. `C:\test\logo.png` on Windows). [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java import org.openqa.selenium.By; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URL; import java.util.HashMap; import java.util.Map; public class JavaSample { public static void main(String[] args) throws Exception { ChromeOptions options = new ChromeOptions(); options.setPlatformName("WIN11"); options.setBrowserVersion("latest"); Map tbOptions = new HashMap<>(); tbOptions.put("key", "API_KEY"); tbOptions.put("secret", "API_SECRET"); tbOptions.put("name", "Test File Upload"); tbOptions.put("upload", "https://testingbot.com/assets/logo-head.png"); tbOptions.put("uploadFilepath", "C:\\test\\logo.png"); options.setCapability("tb:options", tbOptions); RemoteWebDriver driver = new RemoteWebDriver(new URL("https://hub.testingbot.com/wd/hub"), options); // No LocalFileDetector needed, file is already on the VM driver.get("http://the-internet.herokuapp.com/upload"); driver.findElement(By.id("file-upload")).sendKeys("C:\\test\\logo.png"); driver.findElement(By.id("file-submit")).click(); driver.quit(); } } ``` ```python from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.chrome.options import Options options = Options() options.set_capability('platformName', 'WIN11') options.set_capability('browserVersion', 'latest') options.set_capability('tb:options', { 'key': 'API_KEY', 'secret': 'API_SECRET', 'name': 'Test File Upload', 'upload': 'https://testingbot.com/assets/logo-head.png', 'uploadFilepath': 'C:\\test\\logo.png' }) driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options) driver.get('http://the-internet.herokuapp.com/upload') driver.find_element(By.ID, 'file-upload').send_keys('C:\\test\\logo.png') driver.find_element(By.ID, 'file-submit').click() driver.quit() ``` ```javascript const { Builder, By } = require('selenium-webdriver'); const chrome = require('selenium-webdriver/chrome'); async function runTest() { const options = new chrome.Options(); options.set('platformName', 'WIN11'); options.set('browserVersion', 'latest'); options.set('tb:options', { 'key': 'API_KEY', 'secret': 'API_SECRET', 'name': 'Test File Upload', 'upload': 'https://testingbot.com/assets/logo-head.png', 'uploadFilepath': 'C:\\test\\logo.png' }); const driver = await new Builder() .usingServer('https://hub.testingbot.com/wd/hub') .setChromeOptions(options) .build(); // No FileDetector needed, file is already on the VM try { await driver.get('http://the-internet.herokuapp.com/upload'); await driver.findElement(By.id('file-upload')).sendKeys('C:\\test\\logo.png'); await driver.findElement(By.id('file-submit')).click(); console.log(await driver.getTitle()); } finally { await driver.quit(); } } runTest(); ``` ```ruby require 'rubygems' require 'test/unit' require 'selenium-webdriver' class UploadTest < Test::Unit::TestCase def setup options = Selenium::WebDriver::Chrome::Options.new options.add_option('platformName', 'WIN11') options.add_option('browserVersion', 'latest') options.add_option('tb:options', { 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'Test File Upload', 'upload' => 'https://testingbot.com/assets/logo-head.png', 'uploadFilepath' => 'C:\\test\\logo.png' }) @driver = Selenium::WebDriver.for( :remote, url: 'https://hub.testingbot.com/wd/hub', options: options ) end def test_upload @driver.navigate.to 'http://the-internet.herokuapp.com/upload' element = @driver.find_element(:id, 'file-upload') element.send_keys 'C:\\test\\logo.png' @driver.find_element(:id, 'file-submit').click assert_equal 'logo.png', @driver.find_element(:id, 'uploaded-files').text end def teardown @driver.quit end end ``` ```php setCapability('platformName', 'WIN11'); $capabilities->setCapability('browserVersion', 'latest'); $capabilities->setCapability('tb:options', [ 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'Test File Upload', 'upload' => 'https://testingbot.com/assets/logo-head.png', 'uploadFilepath' => 'C:\\test\\logo.png' ]); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); $driver = RemoteWebDriver::create( 'https://hub.testingbot.com/wd/hub', $capabilities ); $driver->get('http://the-internet.herokuapp.com/upload'); // No LocalFileDetector needed, file is already on the VM $file_input = $driver->findElement(WebDriverBy::id('file-upload')); $file_input->sendKeys('C:\\test\\logo.png'); $driver->findElement(WebDriverBy::id('file-submit'))->click(); $driver->quit(); ?> ``` ```csharp using System; using System.Collections.Generic; using OpenQA.Selenium; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Remote; namespace SeleniumTest { class Program { static void Main(string[] args) { ChromeOptions options = new ChromeOptions(); options.PlatformName = "WIN11"; options.BrowserVersion = "latest"; var tbOptions = new Dictionary { ["key"] = "API_KEY", ["secret"] = "API_SECRET", ["name"] = "Test File Upload", ["upload"] = "https://testingbot.com/assets/logo-head.png", ["uploadFilepath"] = "C:\\test\\logo.png" }; options.AddAdditionalOption("tb:options", tbOptions); // No LocalFileDetector needed, file is already on the VM IWebDriver driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), options); driver.Navigate().GoToUrl("http://the-internet.herokuapp.com/upload"); IWebElement upload = driver.FindElement(By.Id("file-upload")); upload.SendKeys("C:\\test\\logo.png"); driver.FindElement(By.Id("file-submit")).Click(); driver.Quit(); } } } ``` ## 2. Upload a file during your test You can use `LocalFileDetector` to upload a file from your local computer to the remote VM/device during your test. The file is automatically transferred over the WebDriver protocol before being sent to the file input element. **Important:** You must set `LocalFileDetector` on the driver **before** calling `sendKeys` on the file input. Without it, Selenium will try to find the file path on the remote machine instead of your local machine. In the example below, we upload an image residing on your own computer (`/Users/test/logo.png`) via the remote VM/device in the TestingBot cloud to a website. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java import org.openqa.selenium.By; import org.openqa.selenium.WebDriver; import org.openqa.selenium.WebElement; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.LocalFileDetector; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URL; import java.util.HashMap; import java.util.Map; public class JavaSample { public static void main(String[] args) throws Exception { ChromeOptions options = new ChromeOptions(); options.setPlatformName("WIN11"); options.setBrowserVersion("latest"); Map tbOptions = new HashMap<>(); tbOptions.put("key", "API_KEY"); tbOptions.put("secret", "API_SECRET"); tbOptions.put("name", "Test File Upload"); options.setCapability("tb:options", tbOptions); RemoteWebDriver driver = new RemoteWebDriver(new URL("https://hub.testingbot.com/wd/hub"), options); // Must set LocalFileDetector before uploading driver.setFileDetector(new LocalFileDetector()); driver.get("http://the-internet.herokuapp.com/upload"); driver.findElement(By.id("file-upload")).sendKeys("/Users/test/logo.png"); driver.findElement(By.id("file-submit")).click(); driver.quit(); } } ``` ```python from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.chrome.options import Options options = Options() options.set_capability('platformName', 'WIN11') options.set_capability('browserVersion', 'latest') options.set_capability('tb:options', { 'key': 'API_KEY', 'secret': 'API_SECRET', 'name': 'Test File Upload' }) driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options) # Python's Remote WebDriver uses LocalFileDetector by default driver.get('http://the-internet.herokuapp.com/upload') driver.find_element(By.ID, 'file-upload').send_keys('/Users/test/logo.png') driver.find_element(By.ID, 'file-submit').click() driver.quit() ``` ```javascript const { Builder, By } = require('selenium-webdriver'); const chrome = require('selenium-webdriver/chrome'); const remote = require('selenium-webdriver/remote'); async function runTest() { const options = new chrome.Options(); options.set('platformName', 'WIN11'); options.set('browserVersion', 'latest'); options.set('tb:options', { 'key': 'API_KEY', 'secret': 'API_SECRET', 'name': 'Test File Upload' }); const driver = await new Builder() .usingServer('https://hub.testingbot.com/wd/hub') .setChromeOptions(options) .build(); // Must set FileDetector for remote file uploads driver.setFileDetector(new remote.FileDetector()); try { await driver.get('http://the-internet.herokuapp.com/upload'); await driver.findElement(By.id('file-upload')).sendKeys('/Users/test/logo.png'); await driver.findElement(By.id('file-submit')).click(); console.log(await driver.getTitle()); } finally { await driver.quit(); } } runTest(); ``` ```ruby require 'rubygems' require 'test/unit' require 'selenium-webdriver' class UploadTest < Test::Unit::TestCase def setup options = Selenium::WebDriver::Chrome::Options.new options.add_option('platformName', 'WIN11') options.add_option('browserVersion', 'latest') options.add_option('tb:options', { 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'Test File Upload' }) @driver = Selenium::WebDriver.for( :remote, url: 'https://hub.testingbot.com/wd/hub', options: options ) # Ruby's Remote WebDriver uses LocalFileDetector by default end def test_upload @driver.navigate.to 'http://the-internet.herokuapp.com/upload' element = @driver.find_element(:id, 'file-upload') element.send_keys '/Users/test/logo.png' @driver.find_element(:id, 'file-submit').click assert_equal 'logo.png', @driver.find_element(:id, 'uploaded-files').text end def teardown @driver.quit end end ``` ```php setCapability('platformName', 'WIN11'); $capabilities->setCapability('browserVersion', 'latest'); $capabilities->setCapability('tb:options', [ 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'Test File Upload' ]); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); $driver = RemoteWebDriver::create( 'https://hub.testingbot.com/wd/hub', $capabilities ); // Must set LocalFileDetector on the driver before uploading $driver->setFileDetector(new LocalFileDetector()); $driver->get('http://the-internet.herokuapp.com/upload'); $file_input = $driver->findElement(WebDriverBy::id('file-upload')); $file_input->sendKeys('/Users/test/logo.png'); $driver->findElement(WebDriverBy::id('file-submit'))->click(); $driver->quit(); ?> ``` ```csharp using System; using System.Collections.Generic; using OpenQA.Selenium; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Remote; namespace SeleniumTest { class Program { static void Main(string[] args) { ChromeOptions options = new ChromeOptions(); options.PlatformName = "WIN11"; options.BrowserVersion = "latest"; var tbOptions = new Dictionary { ["key"] = "API_KEY", ["secret"] = "API_SECRET", ["name"] = "Test File Upload" }; options.AddAdditionalOption("tb:options", tbOptions); IWebDriver driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), options); // Must set LocalFileDetector before uploading var allowsDetection = driver as IAllowsFileDetection; allowsDetection.FileDetector = new LocalFileDetector(); driver.Navigate().GoToUrl("http://the-internet.herokuapp.com/upload"); IWebElement upload = driver.FindElement(By.Id("file-upload")); upload.SendKeys("/Users/test/logo.png"); driver.FindElement(By.Id("file-submit")).Click(); driver.Quit(); } } } ``` ## 3. Upload with WebdriverIO WebdriverIO handles file uploads differently from raw Selenium. It provides a built-in `uploadFile` command on the browser object that transfers a local file to the remote machine and returns the remote path. ### Upload a file before your test starts You can use the `upload` and `uploadFilepath` capabilities with WebdriverIO, just like with Selenium. ```javascript // wdio.conf.js exports.config = { user: 'API_KEY', key: 'API_SECRET', capabilities: [{ browserName: 'chrome', browserVersion: 'latest', platformName: 'WIN11', 'tb:options': { name: 'WebdriverIO File Upload', upload: 'https://testingbot.com/assets/logo-head.png', uploadFilepath: 'C:\\test\\logo.png' } }], hostname: 'hub.testingbot.com', port: 443, path: '/wd/hub', protocol: 'https' }; ``` Then in your test, use the pre-loaded file path: ```javascript describe('File Upload', () => { it('should upload a pre-loaded file', async () => { await browser.url('http://the-internet.herokuapp.com/upload'); const fileInput = await $('#file-upload'); await fileInput.setValue('C:\\test\\logo.png'); await $('#file-submit').click(); const uploadedFile = await $('#uploaded-files'); await expect(uploadedFile).toHaveText('logo.png'); }); }); ``` ### Upload a local file during your test WebdriverIO's `browser.uploadFile()` method transfers a file from your local machine to the remote VM and returns the remote file path. Use this path with `setValue()` on the file input. ```javascript describe('File Upload', () => { it('should upload a local file during the test', async () => { // Transfer file to remote machine, returns remote path const remotePath = await browser.uploadFile('/Users/test/logo.png'); await browser.url('http://the-internet.herokuapp.com/upload'); const fileInput = await $('#file-upload'); await fileInput.setValue(remotePath); await $('#file-submit').click(); const uploadedFile = await $('#uploaded-files'); await expect(uploadedFile).toHaveText('logo.png'); }); }); ``` The `browser.uploadFile()` command is only available when running tests on a remote Selenium grid (like TestingBot). It will not work when running tests locally. ## 4. TestingBot Storage With [TestingBot Storage](https://testingbot.com/support/api/storage#upload), you can upload your files to our servers ahead of time. The advantage is that our test VMs can immediately download your file from our own network, which is much faster than downloading from the public internet. TestingBot Storage is recommended for large files or when you need consistent upload speeds across test runs. Files are stored securely and available for 60 days. ### Step 1: Upload your file to TestingBot Storage First, upload your file to TestingBot Storage using the [REST API](https://testingbot.com/support/api/storage#upload). The API returns a `tb://` URL that you'll use in your test capabilities. #### Upload with curl ```bash curl -X POST "https://api.testingbot.com/v1/storage" \ -u API_KEY:API_SECRET \ -F "file=@/path/to/your/file.png" ``` The API responds with a JSON object containing the `tb://` URL: ```json {"app_url": "tb://398eijf83i"} ``` You can also upload a file from a remote URL instead of a local file: ```bash curl -X POST "https://api.testingbot.com/v1/storage" \ -u API_KEY:API_SECRET \ -d "url=https://example.com/myfile.png" ``` #### Upload with an API client [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) ```java import com.testingbot.TestingbotREST; import com.testingbot.models.TestingbotStorageUploadResponse; import java.io.File; TestingbotREST restApi = new TestingbotREST( "API_KEY", "API_SECRET"); // Upload a local file TestingbotStorageUploadResponse response = restApi.uploadToStorage( new File("/path/to/your/file.png")); // response.getAppUrl() returns "tb://398eijf83i" String tbUrl = response.getAppUrl(); ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient( 'API_KEY', 'API_SECRET') # Upload a local file response = tb.storage.upload_local_file('/path/to/your/file.png') # response['app_url'] returns "tb://398eijf83i" tb_url = response['app_url'] ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: 'API_KEY', api_secret: 'API_SECRET' }); // Upload a local file api.uploadFile('/path/to/your/file.png', function(error, response) { // response.app_url returns "tb://398eijf83i" console.log(response.app_url); }); ``` ```ruby require 'testingbot' api = TestingBot::Api.new( 'API_KEY', 'API_SECRET') # Upload a local file response = api.upload_local_file('/path/to/your/file.png') # response['app_url'] returns "tb://398eijf83i" tb_url = response['app_url'] ``` ```php $api = new TestingBot\TestingBotAPI( 'API_KEY', 'API_SECRET'); // Upload a local file $response = $api->uploadLocalFileToStorage('/path/to/your/file.png'); // $response['app_url'] returns "tb://398eijf83i" $tbUrl = $response['app_url']; ``` ### Step 2: Use the `tb://` URL in your test Once you have the `tb://` URL, use it as the `upload` value in your `tb:options` capabilities. TestingBot will fetch the file from internal storage and place it at the `uploadFilepath` location on the test VM. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) [WebdriverIO](https://testingbot.com#) ```java import org.openqa.selenium.By; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URL; import java.util.HashMap; import java.util.Map; public class StorageUploadTest { public static void main(String[] args) throws Exception { ChromeOptions options = new ChromeOptions(); options.setPlatformName("WIN11"); options.setBrowserVersion("latest"); Map tbOptions = new HashMap<>(); tbOptions.put("key", "API_KEY"); tbOptions.put("secret", "API_SECRET"); tbOptions.put("name", "Test File Upload via Storage"); // Use the tb:// URL from Step 1 tbOptions.put("upload", "tb://398eijf83i"); tbOptions.put("uploadFilepath", "C:\\test\\file.png"); options.setCapability("tb:options", tbOptions); RemoteWebDriver driver = new RemoteWebDriver( new URL("https://hub.testingbot.com/wd/hub"), options); driver.get("http://the-internet.herokuapp.com/upload"); driver.findElement(By.id("file-upload")).sendKeys("C:\\test\\file.png"); driver.findElement(By.id("file-submit")).click(); driver.quit(); } } ``` ```python from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.chrome.options import Options options = Options() options.set_capability('platformName', 'WIN11') options.set_capability('browserVersion', 'latest') options.set_capability('tb:options', { 'key': 'API_KEY', 'secret': 'API_SECRET', 'name': 'Test File Upload via Storage', # Use the tb:// URL from Step 1 'upload': 'tb://398eijf83i', 'uploadFilepath': 'C:\\test\\file.png' }) driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options) driver.get('http://the-internet.herokuapp.com/upload') driver.find_element(By.ID, 'file-upload').send_keys('C:\\test\\file.png') driver.find_element(By.ID, 'file-submit').click() driver.quit() ``` ```javascript const { Builder, By } = require('selenium-webdriver'); const chrome = require('selenium-webdriver/chrome'); async function runTest() { const options = new chrome.Options(); options.set('platformName', 'WIN11'); options.set('browserVersion', 'latest'); options.set('tb:options', { 'key': 'API_KEY', 'secret': 'API_SECRET', 'name': 'Test File Upload via Storage', // Use the tb:// URL from Step 1 'upload': 'tb://398eijf83i', 'uploadFilepath': 'C:\\test\\file.png' }); const driver = await new Builder() .usingServer('https://hub.testingbot.com/wd/hub') .setChromeOptions(options) .build(); try { await driver.get('http://the-internet.herokuapp.com/upload'); await driver.findElement(By.id('file-upload')).sendKeys('C:\\test\\file.png'); await driver.findElement(By.id('file-submit')).click(); } finally { await driver.quit(); } } runTest(); ``` ```ruby require 'rubygems' require 'test/unit' require 'selenium-webdriver' class UploadTest < Test::Unit::TestCase def setup options = Selenium::WebDriver::Chrome::Options.new options.add_option('platformName', 'WIN11') options.add_option('browserVersion', 'latest') options.add_option('tb:options', { 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'Test File Upload via Storage', # Use the tb:// URL from Step 1 'upload' => 'tb://398eijf83i', 'uploadFilepath' => 'C:\\test\\file.png' }) @driver = Selenium::WebDriver.for( :remote, url: 'https://hub.testingbot.com/wd/hub', options: options ) end def test_upload @driver.navigate.to 'http://the-internet.herokuapp.com/upload' element = @driver.find_element(:id, 'file-upload') element.send_keys 'C:\\test\\file.png' @driver.find_element(:id, 'file-submit').click assert_equal 'file.png', @driver.find_element(:id, 'uploaded-files').text end def teardown @driver.quit end end ``` ```php setCapability('platformName', 'WIN11'); $capabilities->setCapability('browserVersion', 'latest'); // Use the tb:// URL from Step 1 $capabilities->setCapability('tb:options', [ 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'Test File Upload via Storage', 'upload' => 'tb://398eijf83i', 'uploadFilepath' => 'C:\\test\\file.png' ]); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); $driver = RemoteWebDriver::create( 'https://hub.testingbot.com/wd/hub', $capabilities ); $driver->get('http://the-internet.herokuapp.com/upload'); $driver->findElement(WebDriverBy::id('file-upload'))->sendKeys('C:\\test\\file.png'); $driver->findElement(WebDriverBy::id('file-submit'))->click(); $driver->quit(); ?> ``` ```csharp using System; using System.Collections.Generic; using OpenQA.Selenium; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Remote; namespace SeleniumTest { class Program { static void Main(string[] args) { ChromeOptions options = new ChromeOptions(); options.PlatformName = "WIN11"; options.BrowserVersion = "latest"; var tbOptions = new Dictionary { ["key"] = "API_KEY", ["secret"] = "API_SECRET", ["name"] = "Test File Upload via Storage", // Use the tb:// URL from Step 1 ["upload"] = "tb://398eijf83i", ["uploadFilepath"] = "C:\\test\\file.png" }; options.AddAdditionalOption("tb:options", tbOptions); IWebDriver driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), options); driver.Navigate().GoToUrl("http://the-internet.herokuapp.com/upload"); driver.FindElement(By.Id("file-upload")).SendKeys("C:\\test\\file.png"); driver.FindElement(By.Id("file-submit")).Click(); driver.Quit(); } } } ``` ```javascript // wdio.conf.js exports.config = { user: 'API_KEY', key: 'API_SECRET', capabilities: [{ browserName: 'chrome', browserVersion: 'latest', platformName: 'WIN11', 'tb:options': { name: 'Test File Upload via Storage', // Use the tb:// URL from Step 1 upload: 'tb://398eijf83i', uploadFilepath: 'C:\\test\\file.png' } }], hostname: 'hub.testingbot.com', port: 443, path: '/wd/hub', protocol: 'https' }; ``` ## Tips & Best Practices ### Always set LocalFileDetector for remote uploads When running tests on a remote grid, `sendKeys` on a file input will look for the file on the **remote** machine by default. Setting `LocalFileDetector` tells Selenium to transfer the file from your local machine first. Forgetting to set `LocalFileDetector` is the most common cause of file upload failures on remote grids. The error message may not clearly indicate this, you'll typically see a "file not found" error on the remote machine. ### Handle hidden file inputs Many modern web applications hide the native `` element and use a styled button or drag-and-drop zone instead. You can still use `sendKeys` on hidden file inputs, Selenium allows interaction with hidden file input elements. ```java // The file input may be hidden, but sendKeys still works WebElement fileInput = driver.findElement(By.cssSelector("input[type='file']")); fileInput.sendKeys("/path/to/file.png"); ``` ### Wait for the upload to complete After triggering a file upload, always wait for a confirmation element or success message before proceeding. File transfers take time, especially for larger files. ```java // Wait for a success indicator after uploading WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(30)); wait.until(ExpectedConditions.visibilityOfElementLocated( By.id("uploaded-files"))); ``` ### Use absolute file paths Always use absolute file paths (e.g. `/Users/test/logo.png` or `C:\test\logo.png`) instead of relative paths. Relative paths may resolve differently on the remote machine and your local machine. ### Upload multiple files If the file input accepts multiple files (has the `multiple` attribute), you can upload several files by sending all paths separated by a newline character (`\n`). ```python # Upload multiple files at once file_input = driver.find_element(By.CSS_SELECTOR, 'input[type="file"]') file_input.send_keys('/Users/test/file1.png\n/Users/test/file2.png') ``` ## Frequently Asked Questions ### Why does my file upload fail with "file not found" on the remote machine? This happens when `LocalFileDetector` is not set. Without it, Selenium sends the local file path as a string to the remote machine, which then tries to find the file at that path locally. Set `LocalFileDetector` on the driver before calling `sendKeys`. Python and Ruby remote drivers use `LocalFileDetector` by default. ### Can I upload a file to a drag-and-drop upload zone? Drag-and-drop upload zones typically have a hidden `` element underneath. Locate this hidden input using `By.cssSelector("input[type='file']")` and call `sendKeys` on it directly. This bypasses the drag-and-drop interface and triggers the same upload behavior. ### What is the difference between uploading before and during a test? Uploading **before** the test (via the `upload` capability) downloads the file from a URL and places it on the VM before your test starts. This is useful for test fixtures and large files. Uploading **during** the test (via `LocalFileDetector`) transfers a file from your local machine through the WebDriver protocol as part of the test flow. Use this when you need to test the actual upload UI interaction. ### How do I upload files with WebdriverIO on TestingBot? WebdriverIO provides a built-in `browser.uploadFile()` command that transfers a local file to the remote machine and returns the remote path. Use this path with `setValue()` on the file input element. Alternatively, use the `upload` and `uploadFilepath` capabilities in your `wdio.conf.js` to pre-load files before the test. See the [WebdriverIO section](https://testingbot.com#webdriverio) for full examples. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/test-options # Selenium Test Options This page offers an overview of all the various options you can specify when starting an automated Selenium test. The **Selenium-Specific Settings** are required to run automated Selenium tests: `browserName`, `browserVersion` and `platformName`. The other options are **TestingBot Options** , these allow you to customize your test in terms of specific driver versions, privacy options, platform options and more. The [Selenium Capabilities generator](https://testingbot.com/support/web-automate/selenium/capabilities) allows you to easily generate the necessary capabilities for your tests. ## Required Selenium Settings ### Browser Name The name of the browser to run your automated test on. Please see our [list of browsers](https://testingbot.com/support/web-automate/browsers) that we support. The `browserName` is a required field that needs to be passed to us via Selenium's [Desired Capabilities](https://github.com/SeleniumHQ/selenium/wiki/DesiredCapabilities). [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["browserName"] = "chrome" ``` ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("browserName", "chrome"); ``` ```php $caps = array( "browserName" => "chrome" ); ``` ```python capabilities = { "browserName" : "chrome" } ``` ```javascript const capabilities = { "browserName" : "chrome" } ``` ```csharp DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("browserName", "chrome"); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { browserName: "chrome" } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "chrome"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'name': 'W3C Sample' } chromeOpts = { 'browserName': "chrome", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', /** Google requires "w3c" to be set in "goog:chromeOptions" as true if you're using ChromeDriver version 74 or lower. * Based on this commit: https://chromium.googlesource.com/chromium/src/+/2b49880e2481658e0702fd6fe494859bca52b39c * ChromeDriver now uses w3c by default from version 75+ so setting this option will no longer be a requirement **/ "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret" } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["name"] = TestContext.CurrentContext.Test.Name }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` ### Browser Version The version of the browser to run your automated test on. Please see our [list of browsers + versions](https://testingbot.com/support/web-automate/browsers) that we support. The `version` is a required field that needs to be passed to us via Selenium's [Desired Capabilities](https://github.com/SeleniumHQ/selenium/wiki/DesiredCapabilities). [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["version"] = "81" ``` ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("version", "81"); ``` ```php $caps = array( "version" => "81" ); ``` ```python capabilities = { "version" : "81" } ``` ```javascript const capabilities = { "version" : "81" } ``` ```csharp DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("version", "81"); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { browserName: "chrome", browserVersion: "latest" } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("browserVersion", "latest"); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "chrome"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setVersion(81); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'name': 'W3C Sample' } chromeOpts = { 'browserName': "chrome", 'browserVersion': 'latest', 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "browserVersion": "latest", /** Google requires "w3c" to be set in "goog:chromeOptions" as true if you're using ChromeDriver version 74 or lower. * Based on this commit: https://chromium.googlesource.com/chromium/src/+/2b49880e2481658e0702fd6fe494859bca52b39c * ChromeDriver now uses w3c by default from version 75+ so setting this option will no longer be a requirement **/ "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret" } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["name"] = TestContext.CurrentContext.Test.Name }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` Other Options: - `"version": "*"` : If you use `*` as version, TestingBot will pick a random version. - `"version": "latest"` : TestingBot will automatically take the latest version. You can also use `latest-1`, `latest-2`, ... to test on the next most recent versions. For example, if the current latest Firefox version is 81 and you use `latest-2`, then the test will run on Firefox 79. - `"version": "<=16"` : TestingBot will pick a version smaller than or equal to the version you specify with `<=`. `"version": "16>="` : TestingBot will pick a version higher than or equal to the version you specify with `>=`. ### Platform Indicates on which operating system the test should run. Please see a [list of platforms](https://testingbot.com/support/web-automate/browsers) that we currently support. The `platform` is a required field that needs to be passed to us via Selenium's [Desired Capabilities](https://github.com/SeleniumHQ/selenium/wiki/DesiredCapabilities). [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["platform"] = "WIN11" ``` ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("platform", "WIN11"); ``` ```php $caps = array( "platform" => "WIN11" ); ``` ```python capabilities = { "platform" : "WIN11" } ``` ```javascript const capabilities = { "platform" : "WIN11" } ``` ```csharp DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("platform", "WIN11"); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platform_name: "WIN11" } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "WIN11"); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "chrome"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('WIN11'); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'name': 'W3C Sample' } chromeOpts = { 'browserName': "chrome", 'platformName': "WIN11", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "platformName": 'WIN11', /** Google requires "w3c" to be set in "goog:chromeOptions" as true if you're using ChromeDriver version 74 or lower. * Based on this commit: https://chromium.googlesource.com/chromium/src/+/2b49880e2481658e0702fd6fe494859bca52b39c * ChromeDriver now uses w3c by default from version 75+ so setting this option will no longer be a requirement **/ "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret" } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["name"] = TestContext.CurrentContext.Test.Name }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` ### Device Name When running a mobile automated test, you'll need to specify on which [mobile device](https://testingbot.com/support/app-automate/devices) (or [simulator/emulator](https://testingbot.com/support/web-automate/browsers)) you want to test. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { deviceName: "Pixel 9" } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "Android"); caps.setCapability("tb:options", tbOptions); caps.setCapability("appium:deviceName", "Pixel 9"); ``` ```python tbOptions = { 'name': 'W3C Sample' } chromeOpts = { 'deviceName': "Pixel 9", 'platformName': "Android", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "appium:deviceName": 'Pixel 9', "platformName": 'Android', "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret" } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { PlatformName = "Android", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["name"] = TestContext.CurrentContext.Test.Name, ["appium:deviceName"] = "Pixel 9" }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` We offer regex/wildcard parameters which you can use to allocate a device: | Regex Input | Result | | --- | --- | | `"iPhone.*"` | This will allocate any available iPhone device (phone) | | `".*Galaxy.*"` | This will allocate any of the available Galaxy devices (phone or tablet) | | `"*"` | This will allocate a random available device, either iOS or Android device | | `"iPhone [8-11]"` | This will allocate either an iPhone 8 or 11 | | `"iPhone 6.*"` | This will allocate either an iPhone 6 or 6S | Some Examples: ```javascript // find any iPhone, except 6 or 6s capabilities.setCapability("appium:deviceName", "^(iPhone.*)(?!6|6S)$"); // find any device which name starts with Galaxy capabilities.setCapability("appium:deviceName", "Galaxy.*"); ``` ### Device Version You can specify the version of the device you want to target. In case of physical devices, you might want to use regex/wildcard patterns to target a broad range of devices. This way your chance of hitting an occupied device during your automated test decreases significantly. Instead of only targeting a specific iOS or Android device, you can target a wider range of devices. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { version: "(16|17).*" } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("appium:platformName", "Android"); caps.setCapability("appium:deviceName", "*"); caps.setCapability("tb:options", tbOptions); caps.setCapability("appium:version", "(13|14).*"); ``` ```python tbOptions = { 'name': 'W3C Sample' } chromeOpts = { 'appium:deviceName': "*", 'appium:version': '(13|14).*', 'platformName': "Android", 'goog:chromeOptions': {'w3c': True} } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "appium:deviceName": '*', "appium:version": '(13|14).*', "platformName": 'Android', "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret" } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { PlatformName = "Android" }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["name"] = TestContext.CurrentContext.Test.Name, ["deviceName"] = "*", ["version"] = "(13|14).*" }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` Some Examples: ```javascript // target multiple specific versions of an iPhone device capabilities.setCapability("appium:deviceName", "iPhone.*"); capabilities.setCapability("appium:version", "15.0|14.2|14|13.4|13.3"); // find any Galaxy device with Android version 13 or 14 capabilities.setCapability("appium:deviceName", "Galaxy.*"); capabilities.setCapability("appium:version", "(13|14).*"); ``` ### Tablet Only You can specify this capability when you only want to allocate a tablet device. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { deviceName: "*", tabletOnly: true } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("tabletOnly", true); tbOptions.setCapability("deviceName", "*"); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "Android"); caps.setCapability("tb:options", tbOptions); ``` ```python tbOptions = { 'name': 'W3C Sample' } chromeOpts = { 'deviceName': "*", 'tabletOnly': true, 'platformName': "Android", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "deviceName": '*', "platformName": 'Android', "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "tabletOnly": true } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { PlatformName = "Android", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["name"] = TestContext.CurrentContext.Test.Name, ["deviceName"] = "*", ["tabletOnly"] = true }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` ### Phone Only You can specify this capability when you only want to allocate a phone device. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { deviceName: "*", phoneOnly: true } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("phoneOnly", true); tbOptions.setCapability("deviceName", "*"); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "Android"); caps.setCapability("tb:options", tbOptions); ``` ```python tbOptions = { 'name': 'W3C Sample' } chromeOpts = { 'deviceName': "*", 'phoneOnly': true, 'platformName': "Android", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "deviceName": '*', "platformName": 'Android', "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "phoneOnly": true } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { PlatformName = "Android", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["name"] = TestContext.CurrentContext.Test.Name, ["deviceName"] = "*", ["phoneOnly"] = true }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` ### Use a specific version of Selenium By default we use Selenium version `2.53.1` to run your test. If you wish to use another Selenium version for your test, please specify one of the following available versions: [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["selenium-version"] = '2.53.1' ``` ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("selenium-version", "2.53.1"); ``` ```php $caps = array( "selenium-version" => "2.53.1" ); ``` ```python capabilities = { "selenium-version" : '2.53.1' } ``` ```javascript const capabilities = { "selenium-version" : '2.53.1' } ``` ```csharp DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("selenium-version", "2.53.1"); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "WIN11", browserName: "chrome", browserVersion: "latest", "tb:options" => { "selenium-version" : '2.53.1' } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("selenium-version", "2.53.1"); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "WIN11"); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "chrome"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('WIN11'); $capabilities->setCapability('tb:options', array( 'selenium-version' => '2.53.1' )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'name': 'W3C Sample', 'selenium-version': '2.53.1' } chromeOpts = { 'browserName': "chrome", 'platformName': "WIN11", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "platformName": 'WIN11', /** Google requires "w3c" to be set in "goog:chromeOptions" as true if you're using ChromeDriver version 74 or lower. * Based on this commit: https://chromium.googlesource.com/chromium/src/+/2b49880e2481658e0702fd6fe494859bca52b39c * ChromeDriver now uses w3c by default from version 75+ so setting this option will no longer be a requirement **/ "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "selenium-version": '2.53.1' } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["selenium-version"] = "2.53.1" }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` | Value Type | Default Value | Possible Values: | | --- | --- | --- | | string | "2.53.1" | 4.48.0 4.47.0 4.46.0 4.45.0 4.44.0 4.43.0 4.42.0 4.41.0 4.40.0 4.39.0 4.38.0 4.37.0 4.36.0 4.35.0 4.34.0 4.33.0 4.32.0 4.31.0 4.30.0 4.29.0 4.28.1 4.28.0 4.27.0 4.26.0 4.25.0 4.24.0 4.23.1 4.23.0 4.22.0 4.21.0 4.20.0 4.19.0 4.18.1 4.18.0 4.17.0 4.16.1 4.16.0 4.15.0 4.14.1 4.14.0 4.13.0 4.12.1 4.12.0 4.11.0 4.10.0 4.9.0 4.8.3 4.8.2 4.8.1 4.8.0 4.7.0 4.6.0 4.5.3 4.5.2 4.5.1 4.5.0 4.4.0 4.3.0 4.2.2 4.2.1 4.2.0 4.1.3 4.1.2 4.1.1 4.1.0 4.0.0 4.0.0-beta-4 4.0.0-beta-3 4.0.0-beta-2 4.0.0-beta-1 4.0.0-alpha-7 4.0.0-alpha-6 3.141.59 3.141.5 3.141.0 3.14.0 3.13.0 3.12.0 3.11.0 3.10.0 3.9.0 3.8.1 3.8.0 3.7.1 3.7.0 3.6.0 3.5.3 3.5.2 3.5.1 3.5.0 3.4.0 3.3.1 3.3.0 3.2.0 3.1.0 3.0.1 2.53.1 2.53.0 2.52.0 2.51.0 2.50.0 2.49.0 2.48.2 2.48.1 2.48.0 | ### Chromedriver We support using custom ChromeDriver versions during your tests. By default the most recent Chromedriver is used according to the version of Chrome you're testing on. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["chromedriverVersion"] = "76.0.3809.25" ``` ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("chromedriverVersion", "76.0.3809.25"); ``` ```php $caps = array( "chromedriverVersion" => "76.0.3809.25" ); ``` ```python capabilities = { "chromedriverVersion" : "76.0.3809.25" } ``` ```javascript const capabilities = { "chromedriverVersion" : "76.0.3809.25" } ``` ```csharp DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("chromedriverVersion", "76.0.3809.25"); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "WIN11", browserName: "chrome", browserVersion: "latest", "tb:options" => { "chromedriverVersion" : "76.0.3809.25" } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("chromedriverVersion", "76.0.3809.25"); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "WIN11"); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "chrome"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('WIN11'); $capabilities->setCapability('tb:options', array( 'chromedriverVersion' => "76.0.3809.25" )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'name': 'W3C Sample', 'chromedriverVersion': "76.0.3809.25" } chromeOpts = { 'browserName': "chrome", 'platformName': "WIN11", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "platformName": 'WIN11', /** Google requires "w3c" to be set in "goog:chromeOptions" as true if you're using ChromeDriver version 74 or lower. * Based on this commit: https://chromium.googlesource.com/chromium/src/+/2b49880e2481658e0702fd6fe494859bca52b39c * ChromeDriver now uses w3c by default from version 75+ so setting this option will no longer be a requirement **/ "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "chromedriverVersion": "76.0.3809.25" } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "WIN11" }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["chromedriverVersion"] = "76.0.3809.25" }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` | Value Type | Default Value | Possible Values: | | --- | --- | --- | | string | "2.46" | dev 149.0.7795.2 148.0.7778.40 147.0.7727.3 146.0.7680.0 145.0.7632.76 145.0.7561.2 144.0.7559.3 143.0.7499.40 142.0.7444.59 141.0.7390.54 140.0.7339.80 139.0.7207.2 138.0.7153.0 137.0.7117.2 136.0.7064.0 135.0.6999.2 134.0.6958.2 133.0.6943.16 132.0.6811.2 131.0.6778.3 130.0.6669.2 129.0.6614.3 128.0.6613.18 127.0.6533.88 127.0.6485.0 126.0.6423.2 125.0.6422.4 124.0.6367.60 124.0.6356.2 123.0.6262.5 122.0.6253.3 121.0.6154.0 121.0.6129.0 120.0.6099.5 119.0.6045.9 118.0.5979.0 117.0.5938.0 116.0.5845.82 115.0.5790.170 115.0.5790.90 114.0.5735.90 114.0.5735.16 113.0.5672.63 113.0.5672.24 112.0.5615.49 112.0.5615.28 111.0.5563.64 111.0.5563.41 111.0.5563.19 110.0.5481.77 110.0.5481.30 109.0.5414.74 109.0.5414.25 108.0.5359.71 108.0.5359.22 107.0.5304.62 107.0.5304.18 106.0.5249.61 106.0.5249.21 105.0.5195.52 105.0.5195.19 104.0.5112.79 104.0.5112.29 104.0.5112.20 103.0.5060.134 103.0.5060.24 102.0.5005.61 102.0.5005.27 101.0.4951.41 101.0.4951.15 100.0.4896.60 100.0.4896.20 99.0.4844.51 99.0.4844.35 99.0.4844.17 98.0.4758.80 98.0.4758.48 97.0.4692.71 97.0.4692.36 97.0.4692.20 96.0.4664.45 95.0.4638.10 94.0.4606.41 93.0.4577.15 92.0.4515.43 91.0.4472.101 91.0.4472.19 90.0.4430.24 89.0.4389.23 88.0.4324.96 88.0.4324.27 87.0.4280.88 87.0.4280.20 86.0.4240.22 85.0.4183.87 85.0.4183.83 85.0.4183.38 84.0.4147.30 83.0.4103.39 83.0.4103.14 81.0.4044.69 81.0.4044.20 80.0.3987.16 79.0.3945.36 79.0.3945.16 78.0.3904.70 78.0.3904.11 77.0.3865.40 77.0.3865.10 76.0.3809.68 76.0.3809.25 76.0.3809.12 75.0.3770.140 75.0.3770.90 75.0.3770.8 74.0.3729.6 73.0.3683.68 73.0.3683.20 72.0.3626.7 72.0.3626.69 71.0.3578.80 71.0.3578.33 71.0.3578.30 71.0.3578.137 70.0.3538.97 70.0.3538.67 70.0.3538.16 2.46 2.45 2.44 2.43 2.42 2.41 2.40 2.39 2.38 2.37 2.36 2.35 2.34 2.33 2.32 2.31 2.30 2.29 2.28 2.27 2.26 2.25 2.24 2.23 2.22 2.21 2.20 2.19 2.16 2.15 2.14 2.13 2.12 2.10 | ### Internet Explorer Driver (IEDriver) We provide both 32-bit and 64-bit versions of the Internet Explorer driver. By default we use the 2.53.1 (32-bit) IEDriver for your tests. We use 32-bit because of a [slow text entry bug](https://github.com/seleniumhq/selenium-google-code-issue-archive/issues/5116) with the 64-bit version. For our screenshots, we use the 64-bit version, because of a [screenshot problem](https://github.com/seleniumhq/selenium-google-code-issue-archive/issues/5876) with the 32-bit version. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["iedriverVersion"] = "2.53.1" ``` ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("iedriverVersion", "2.53.1"); ``` ```php $caps = array( "iedriverVersion" => "2.53.1" ); ``` ```python capabilities = { "iedriverVersion" : "2.53.1" } ``` ```javascript const capabilities = { "iedriverVersion" : "2.53.1" } ``` ```csharp DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("iedriverVersion", "2.53.1"); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "WIN11", browserName: "internet explorer", browserVersion: "latest", "se:ieOptions": {}, "tb:options": { "iedriverVersion" : "4.8.0", "selenium-version": "4.8.0" } } ``` ```java InternetExplorerOptions ieOpts = new InternetExplorerOptions(); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("iedriverVersion", "4.8.0"); tbOptions.setCapability("selenium-version", "4.8.0"); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("se:ieOptions", ieOpts); caps.setCapability("platformName", "WIN11"); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "internet explorer"); ``` ```php $capabilities = DesiredCapabilities::internetExplorer(); $capabilities->setPlatform('WIN11'); $capabilities->setCapability('tb:options', array( 'iedriverVersion' => "4.8.0", 'selenium-version' => "4.8.0" )); $capabilities->setCapability('se:ieOptions', []); ``` ```python tbOptions = { 'name': 'W3C Sample', 'iedriverVersion': '4.8.0', 'selenium-version': '4.8.0' } ieOpts = { 'browserName': "internet explorer", 'platformName': "WIN11", 'se:ieOptions': {}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=ieOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'internet explorer', "platformName": 'WIN11', "se:ieOptions" : {}, "tb:options": { "key": "api_key", "secret": "api_secret", "iedriverVersion": "4.8.0", "selenium-version": "4.8.0" } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var ieOptions = new InternetExplorerOptions() { BrowserVersion = "11", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["iedriverVersion"] = "4.8.0", ["selenium-version"] = "4.8.0" }; ieOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), ieOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` | Value Type | Default Value | Possible Values: | | --- | --- | --- | | string | 2.53.1 | x64\_4.14.0 4.14.0 x64\_4.11.0 4.11.0 x64\_4.10.0 4.10.0 x64\_4.8.1 4.8.1 x64\_4.8.0 4.8.0 x64\_4.7.0 4.7.0 x64\_4.6.0 4.6.0 x64\_4.3.0 4.3.0 x64\_4.2.0 4.2.0 x64\_4.0.0 4.0.0 x64\_3.141.59 3.141.59 x64\_3.141.5 3.141.5 x64\_3.141.0 3.141.0 x64\_3.14.0 3.14.0 x64\_3.13.0 3.13.0 x64\_3.12.0 3.12.0 x64\_3.11.1 3.11.1 x64\_3.11.0 3.11.0 x64\_3.10.0 3.10.0 x64\_3.9.0 3.9.0 x64\_3.8.0 3.8.0 x64\_3.7.0 3.7.0 x64\_3.6.0 3.6.0 x64\_3.5.0 3.5.0 x64\_3.4.0 3.4.0 x64\_3.3.0 3.3.0 x64\_3.2.0 3.2.0 x64\_3.1.0 3.1.0 x64\_2.53.1 2.53.1 x64\_2.53.0 2.53.0 x64\_2.52.2 2.52.2 x64\_2.51.0 2.51.0 x64\_2.50.0 2.50.0 x64\_2.49.0 2.49.0 x64\_2.48.0 2.48.0 x64\_2.47.0 2.47.0 x64\_2.46.0 2.46.0 x64\_2.45.0 2.45.0 x64\_2.42.0 2.42.0 | ### Edge Driver (MicrosoftWebDriver) EdgeDriver is built by Microsoft to automate the Microsoft Edge Browser. By default, we make sure that the most recent fully-compatible EdgeDriver is used for every test running on the Microsoft Edge Browser. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { "browserName" : "MicrosoftEdge", "platform": "WINDOWS" } caps["edgedriverVersion"] = "110" ``` ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("edgedriverVersion", "110"); ``` ```php $caps = array( "edgedriverVersion" => "110" ); ``` ```python capabilities = { "edgedriverVersion" : "110" } ``` ```javascript const capabilities = { "edgedriverVersion" : "110", "ms:edgeOptions": {} } ``` ```csharp var edgeOptions = new EdgeOptions() { BrowserVersion = "11", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["edgeDriverVersion"] = "110", ["selenium-version"] = "4.8.0" }; edgeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), edgeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "WIN11", browserName: "microsoftedge", browserVersion: "latest", "tb:options" => { "edgedriverVersion" : "110" }, "ms:edgeOptions": {} } ``` ```java EdgeOptions edgeOpts = new EdgeOptions(); edgeOpts.setPlatformName("WIN11"); edgeOpts.setBrowserVersion("latest"); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("edgedriverVersion", "110"); edgeOpts.setCapability("tb:options", tbOptions); ``` ```php $capabilities = DesiredCapabilities::microsoftEdge(); $capabilities->setCapability('platformName', 'WIN11'); $capabilities->setCapability('tb:options', array( 'edgedriverVersion' => "110" )); $capabilities->setCapability('ms:edgeOptions', new \stdClass()); ``` ```python tbOptions = { 'name': 'W3C Sample', 'edgedriverVersion': '110' } edgeOpts = { 'browserName': "MicrosoftEdge", 'platformName': "WIN11", 'tb:options': tbOptions, 'ms:edgeOptions': {} } self.driver = webdriver.Remote(remote_url, desired_capabilities=edgeOpts) ``` ```javascript let driver = await new webdriver.Builder().withCapabilities({ "browserName": 'microsoftedge', "platformName": 'WIN11', "ms:edgeOptions" : {}, "tb:options": { "key": "api_key", "secret": "api_secret", "edgedriverVersion": "110" } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var edgeOptions = new EdgeOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["edgedriverVersion"] = "110" }; edgeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), edgeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` | Value Type | Default Value | Possible Values: | | --- | --- | --- | | string | 15063 | 141 140 139 138 137 136 135 134 133 132 131 130 129 128 127 126 125 124 123 122 121 120 119 118 117 116 115 114 113 112 111 110 109 108 107 106 105 104 103 102 101 100 99 98 97 96 95 94 93 92 91 90 89 88 87 86 85 84 83 81 dev insiders 16299 15063 14393 | ### Firefox Driver (Geckodriver) For Firefox 47 and up, webdriver tests on Firefox need to use [Mozilla's GeckoDriver](https://github.com/mozilla/geckodriver). Specify this option to choose which Geckodriver we should use. By default, we use the version that is most compatible with the Firefox version you request. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["geckodriverVersion"] = "0.28.0" ``` ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("geckodriverVersion", "0.28.0"); ``` ```php $caps = array( "geckodriverVersion" => "0.28.0" ); ``` ```python capabilities = { "geckodriverVersion" : "0.28.0" } ``` ```javascript const capabilities = { "geckodriverVersion" : "0.28.0" } ``` ```csharp DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("geckodriverVersion", "0.28.0"); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "WIN11", browserName: "firefox", browserVersion: "latest", "tb:options": { "geckodriverVersion" : "0.28.0" }, "moz:firefoxOptions": {} } ``` ```java FirefoxOptions ffOptions = new FirefoxOptions(); ffOptions.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("geckodriverVersion", "0.28.0"); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(FirefoxOptions.CAPABILITY, ffOptions); caps.setCapability("platformName", "WIN11"); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "firefox"); ``` ```php $options = new FirefoxOptions(); $capabilities = DesiredCapabilities::firefox(); $capabilities->setPlatform('WIN11'); $capabilities->setCapability('tb:options', array( 'geckodriverVersion' => "0.28.0" )); $capabilities->setCapability('moz:firefoxOptions', array()); $capabilities->setCapability(FirefoxOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'name': 'W3C Sample', 'geckodriverVersion': "0.28.0" } chromeOpts = { 'browserName': "firefox", 'platformName': "WIN11", 'tb:options': tbOptions, 'moz:firefoxOptions': {} } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'firefox', "platformName": 'WIN11', "tb:options": { "key": "api_key", "secret": "api_secret", "geckodriverVersion": "0.28.0" }, 'moz:firefoxOptions': {} }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var ffOptions = new FirefoxOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["geckodriverVersion"] = "0.28.0" }; ffOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), ffOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` | Value Type | Default Value | Possible Values: | | --- | --- | --- | | string | 0.28.0 | 0.36.0 0.35.0 0.34.0 0.33.0 0.32.2 0.32.1 0.32.0 0.31.0 0.30.0 0.29.1 0.29.0 0.28.0 0.27.0 0.26.0 0.25.0 0.24.0 0.23.0 0.22.0 0.21.0 0.20.1 0.19.1 0.19.0 0.18.0 0.17.0 0.16.1 0.16.0 0.15.0 0.14.0 0.13.0 0.12.0 0.11.0 0.10.0 0.9.0 0.8.0 0.7.1 0.6.2 | ### Opera Driver Opera uses the [Chromium OperaDriver](https://github.com/operasoftware/operachromiumdriver/releases) to automate Opera Desktop Browsers. Specify this option to choose which OperaDriver we should use. By default, we use the version that is most compatible with the Opera version you request. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["operaDriverVersion"] = "90.0.4430.85" ``` ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("operaDriverVersion", "90.0.4430.85"); ``` ```php $caps = array( "operaDriverVersion" => "90.0.4430.85" ); ``` ```python capabilities = { "operaDriverVersion" : "90.0.4430.85" } ``` ```javascript const capabilities = { "operaDriverVersion" : "90.0.4430.85" } ``` ```csharp DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("operaDriverVersion", "90.0.4430.85"); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "WIN11", browserName: "opera", browserVersion: "latest", "tb:options" => { "operaDriverVersion" : "90.0.4430.85" } } ``` ```java MutableCapabilities caps = new MutableCapabilities(); caps.setCapability("browserName", "opera"); caps.setCapability("platformName", "WIN11"); caps.setCapability("browserVersion", "latest"); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("operaDriverVersion", "90.0.4430.85"); caps.setCapability("tb:options", tbOptions); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('WIN11'); $capabilities->setBrowserName('opera'); $capabilities->setCapability('tb:options', array( 'operaDriverVersion' => "90.0.4430.85" )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'name': 'W3C Sample', 'operaDriverVersion': "90.0.4430.85" } chromeOpts = { 'browserName': "opera", 'platformName': "WIN11", 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'opera', "platformName": 'WIN11', "tb:options": { "key": "api_key", "secret": "api_secret", "operaDriverVersion": "90.0.4430.85" } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var operaOptions = new OperaOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["operaDriverVersion"] = "90.0.4430.85" }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), operaOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` | Value Type | Default Value | Possible Values: | | --- | --- | --- | | string | 145.0.7632.117 | 146.0.7680.178 145.0.7632.117 144.0.7559.173 143.0.7499.194 142.0.7444.243 141.0.7390.125 140.0.7339.249 139.0.7258.156 138.0.7204.251 137.0.7151.122 135.0.7049.115 134.0.6998.205 133.0.6943.143 132.0.6834.209 131.0.6778.86 130.0.6723.137 128.0.6613.162 127.0.6533.120 126.0.6478.127 125.0.6422.143 124.0.6367.62 123.0.6312.59 122.0.6261.95 121.0.6167.140 120.0.6099.200 119.0.6045.124 118.0.5993.89 117.0.5938.132 116.0.5845.97 115.0.5790.171 114.0.5735.110 113.0.5672.127 112.0.5615.87 111.0.5563.65 110.0.5481.100 109.0.5414.120 108.0.5359.99 107.0.5304.88 106.0.5249.119 105.0.5195.102 104.0.5112.81 103.0.5060.66 102.0.5005.61 101.0.4951.64 100.0.4896.127 99.0.4844.51 98.0.4758.82 97.0.4692.71 96.0.4664.45 95.0.4638.54 94.0.4606.61 93.0.4577.63 92.0.4515.107 91.0.4472.77 90.0.4430.85 89.0.4389.82 88.0.4324.104 87.0.4280.67 86.0.4240.80 85.0.4183.102 84.0.4147.89 83.0.4103.97 81.0.4044.113 80.0.3987.100 79.0.3945.79 78.0.3904.87 77.0.3865.120 76.0.3809.132 75.0.3770.100 2.45 2.42 2.41 2.40 2.38 2.37 2.36 2.35 2.33 2.32 2.30 2.229 2.27 2.26 2.25 2.24 2.23 2.22 | ### Taking screenshots during your tests By default we do not capture screenshots at every step of your test. If you wish to take a screenshot for every step, please add this capability (set to `true`) to your request. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["screenshot"] = true ``` ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("screenshot", true); ``` ```php $caps = array( "screenshot" => true ); ``` ```python capabilities = { "screenshot" : True } ``` ```javascript const capabilities = { "screenshot" : true } ``` ```csharp DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("screenshot", true); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "WIN11", browserName: "chrome", browserVersion: "latest", "tb:options" => { screenshot: true } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("screenshot", true); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "WIN11"); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "chrome"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('WIN11'); $capabilities->setCapability('tb:options', array( 'screenshot' => true )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'name': 'W3C Sample', 'screenshot': True } chromeOpts = { 'browserName': "chrome", 'platformName': "WIN11", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "platformName": 'WIN11', /** Google requires "w3c" to be set in "goog:chromeOptions" as true if you're using ChromeDriver version 74 or lower. * Based on this commit: https://chromium.googlesource.com/chromium/src/+/2b49880e2481658e0702fd6fe494859bca52b39c * ChromeDriver now uses w3c by default from version 75+ so setting this option will no longer be a requirement **/ "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "screenshot": true } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["screenshot"] = true }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` | Value Type | Default Value | | --- | --- | | boolean | false | ### Make a video of your tests By default we record a video of your test, which is accessible in the member area. If you do not wish to have this, you can disable it with this option. Recording adds processing overhead for the duration of the session, so disabling it speeds up tests that do not need a video. See [test speed optimization](https://testingbot.com/support/other/test-speed-optimization#screenrecorder). [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["screenrecorder"] = true ``` ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("screenrecorder", true); ``` ```php $caps = array( "screenrecorder" => true ); ``` ```python capabilities = { "screenrecorder" : True } ``` ```javascript const capabilities = { "screenrecorder" : true } ``` ```csharp DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("screenrecorder", true); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "WIN11", browserName: "chrome", browserVersion: "latest", "tb:options" => { screenrecorder: true } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("screenrecorder", true); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "WIN11"); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "chrome"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('WIN11'); $capabilities->setCapability('tb:options', array( 'screenrecorder' => true )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'name': 'W3C Sample', 'screenrecorder': True } chromeOpts = { 'browserName': "chrome", 'platformName': "WIN11", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "platformName": 'WIN11', /** Google requires "w3c" to be set in "goog:chromeOptions" as true if you're using ChromeDriver version 74 or lower. * Based on this commit: https://chromium.googlesource.com/chromium/src/+/2b49880e2481658e0702fd6fe494859bca52b39c * ChromeDriver now uses w3c by default from version 75+ so setting this option will no longer be a requirement **/ "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "screenrecorder": true } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["screenrecorder"] = true }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` | Value Type | Default Value | | --- | --- | | boolean | true | ### Test Privacy Make the test results for this test public so that everyone can access the results. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["public"] = false ``` ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("public", false); ``` ```php $caps = array( "public" => false ); ``` ```python capabilities = { "public" : False } ``` ```javascript const capabilities = { "public" : false } ``` ```csharp DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("public", false); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "WIN11", browserName: "chrome", browserVersion: "latest", "tb:options" => { "public" : false } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("public", false); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "WIN11"); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "chrome"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('WIN11'); $capabilities->setCapability('tb:options', array( 'public' => 130 )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'public': False } chromeOpts = { 'browserName': "chrome", 'platformName': "WIN11", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "platformName": 'WIN11', /** Google requires "w3c" to be set in "goog:chromeOptions" as true if you're using ChromeDriver version 74 or lower. * Based on this commit: https://chromium.googlesource.com/chromium/src/+/2b49880e2481658e0702fd6fe494859bca52b39c * ChromeDriver now uses w3c by default from version 75+ so setting this option will no longer be a requirement **/ "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "public": false } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["public"] = false }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` | Value Type | Default Value | | --- | --- | | boolean | false | ### Blacklist hostnames The hostnames you specify will be pointed to localhost instead of their real destination. This means you can speed up tests by blocking third party content which you don't need and slows down your test. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["blacklist"] = "site1.com,site2.com" ``` ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("blacklist", "site1.com,site2.com"); ``` ```php $caps = array( "blacklist" => "site1.com,site2.com" ); ``` ```python capabilities = { "blacklist" : "site1.com,site2.com" } ``` ```javascript const capabilities = { "blacklist" : "site1.com,site2.com" } ``` ```csharp DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("blacklist", "site1.com,site2.com"); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "WIN11", browserName: "chrome", browserVersion: "latest", "tb:options" => { "blacklist" : "site1.com,site2.com" } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("blacklist", "site1.com,site2.com"); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "WIN11"); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "chrome"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('WIN11'); $capabilities->setCapability('tb:options', array( 'blacklist' => "site1.com,site2.com" )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'blacklist': "site1.com,site2.com" } chromeOpts = { 'browserName': "chrome", 'platformName': "WIN11", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "platformName": 'WIN11', /** Google requires "w3c" to be set in "goog:chromeOptions" as true if you're using ChromeDriver version 74 or lower. * Based on this commit: https://chromium.googlesource.com/chromium/src/+/2b49880e2481658e0702fd6fe494859bca52b39c * ChromeDriver now uses w3c by default from version 75+ so setting this option will no longer be a requirement **/ "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "blacklist": "site1.com,site2.com" } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["blacklist"] = "site1.com,site2.com" }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` | Value Type | | --- | | string (comma-separated) | ### Customize Logging By default, TestingBot records logs of all Selenium actions and its drivers. Set this option to `false` if you don't want TestingBot to record anything (for example, if you have sensitive data). You will not see any test logs in our member dashboard. Set to `strip-parameters` to prevent the POST/GET parameters from being logged on the TestingBot test detail page (does not affect other logs like Selenium logs, Chromedriver logs, ...). [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["recordLogs"] = true ``` ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("recordLogs", true); ``` ```php $caps = array( "recordLogs" => true ); ``` ```python capabilities = { "recordLogs" : True } ``` ```javascript const capabilities = { "recordLogs" : true } ``` ```csharp DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("recordLogs", true); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "WIN11", browserName: "chrome", browserVersion: "latest", "tb:options" => { "recordLogs" : true } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("recordLogs", true); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "WIN11"); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "chrome"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('WIN11'); $capabilities->setCapability('tb:options', array( 'recordLogs' => true )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'recordLogs': True } chromeOpts = { 'browserName': "chrome", 'platformName': "WIN11", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "platformName": 'WIN11', /** Google requires "w3c" to be set in "goog:chromeOptions" as true if you're using ChromeDriver version 74 or lower. * Based on this commit: https://chromium.googlesource.com/chromium/src/+/2b49880e2481658e0702fd6fe494859bca52b39c * ChromeDriver now uses w3c by default from version 75+ so setting this option will no longer be a requirement **/ "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "recordLogs": true } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["recordLogs"] = true }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` | Value Type | Default Value | Possible Values: | | --- | --- | --- | | string | "true" | `true`, `false`, or `strip-parameters` | ### Custom Time Zones Change the Time Zone of the Virtual Machine to the Time Zone you specify. You can find a [list of timezones on Wikipedia](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones). Only location names are supported (not their paths). See some examples below: [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["timeZone"] = "Etc/UTC" ``` ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("timeZone", "Etc/UTC"); ``` ```php $caps = array( "timeZone" => "Etc/UTC" ); ``` ```python capabilities = { "timeZone" : "Etc/UTC" } ``` ```javascript const capabilities = { "timeZone" : "Etc/UTC" } ``` ```csharp DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("timeZone", "Etc/UTC"); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "WIN11", browserName: "chrome", browserVersion: "latest", "tb:options" => { "timeZone" : "Etc/UTC" } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("timeZone", "Etc/UTC"); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "WIN11"); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "chrome"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('WIN11'); $capabilities->setCapability('tb:options', array( 'timeZone' => "Etc/UTC" )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'timeZone': "Etc/UTC" } chromeOpts = { 'browserName': "chrome", 'platformName': "WIN11", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "platformName": 'WIN11', /** Google requires "w3c" to be set in "goog:chromeOptions" as true if you're using ChromeDriver version 74 or lower. * Based on this commit: https://chromium.googlesource.com/chromium/src/+/2b49880e2481658e0702fd6fe494859bca52b39c * ChromeDriver now uses w3c by default from version 75+ so setting this option will no longer be a requirement **/ "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "timeZone": "Etc/UTC" } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["timeZone"] = "Etc/UTC" }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` | Value Type | Default Value | Possible Values: | | --- | --- | --- | | string | "Etc/UTC" | [List of timezones](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) | ### Change Screen Resolution Will adjust the screen resolution during your test. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["screen-resolution"] = "1280x1024" ``` ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("screen-resolution", "1280x1024"); ``` ```php $caps = array( "screen-resolution" => "1280x1024" ); ``` ```python capabilities = { "screen-resolution" : "1280x1024" } ``` ```javascript const capabilities = { "screen-resolution" : "1280x1024" } ``` ```csharp DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("screen-resolution", "1280x1024"); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "WIN11", browserName: "chrome", browserVersion: "latest", "tb:options" => { "screen-resolution" : "1280x1024" } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("screen-resolution", "1280x1024"); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "WIN11"); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "chrome"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('WIN11'); $capabilities->setCapability('tb:options', array( 'screen-resolution' => "1280x1024" )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'screen-resolution': "My Test Name" } chromeOpts = { 'browserName': "chrome", 'platformName': "WIN11", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "platformName": 'WIN11', /** Google requires "w3c" to be set in "goog:chromeOptions" as true if you're using ChromeDriver version 74 or lower. * Based on this commit: https://chromium.googlesource.com/chromium/src/+/2b49880e2481658e0702fd6fe494859bca52b39c * ChromeDriver now uses w3c by default from version 75+ so setting this option will no longer be a requirement **/ "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "screen-resolution": "1280x1024" } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["screen-resolution"] = "1280x1024" }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` | Value Type | Default Value | | | string | "1280x1024" | \| Platform \| Resolutions \| \| --- \| --- \| \| Windows/Linux \| 800x600 1024x768 1152x864 1280x768 1280x800 1280x960 1280x1024 1400x1050 1600x1200 1680x1050 1920x1080 1920x1200 2560x1440 \| \| macOS \| 800x600 1024x768 1280x768 1280x800 1280x960 1280x1024 1366x768 1440x900 1600x900 1600x1200 1680x1050 1920x1080 1920x1200 2048x1536 \| | ### Customize OS When you specify a prerun file, we will first download the file and execute it (with optional prerun-arguments). This is useful when you want to customize/add software before your test starts. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["prerun"] = "https://..." ``` ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("prerun", "https://..."); ``` ```php $caps = array( "prerun" => "https://..." ); ``` ```python capabilities = { "prerun" : "https://..." } ``` ```javascript const capabilities = { "prerun" : "https://..." } ``` ```csharp DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("prerun", "https://..."); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "WIN11", browserName: "chrome", browserVersion: "latest", "tb:options" => { "prerun" : "https://..." } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("prerun", "https://..."); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "WIN11"); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "chrome"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('WIN11'); $capabilities->setCapability('tb:options', array( 'prerun' => "https://..." )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'prerun': "https://..." } chromeOpts = { 'browserName': "chrome", 'platformName': "WIN11", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "platformName": 'WIN11', /** Google requires "w3c" to be set in "goog:chromeOptions" as true if you're using ChromeDriver version 74 or lower. * Based on this commit: https://chromium.googlesource.com/chromium/src/+/2b49880e2481658e0702fd6fe494859bca52b39c * ChromeDriver now uses w3c by default from version 75+ so setting this option will no longer be a requirement **/ "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "prerun": "https://..." } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["prerun"] = "https://..." }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` | Value Type | | --- | | string (URL) | ```javascript "prerun-args" : "-a -b -c ..." ``` | Value Type | | --- | | string | With [TestingBot Storage](https://testingbot.com/support/api/storage#upload), you can upload your executable on our servers. The advantage of this is that our test VMs can immediately download your executable from our own network, which is much faster than downloading from the public internet. Once the file is uploaded via TestingBot Storage, you can use `"prerun" : "tb://..."` #### Supported file types We detect what you send and run it the right way. Supported types: - `.exe` — executed directly - `.msi` — installed silently via `msiexec /i /quiet /norestart` - `.bat` / `.cmd` — run via `cmd` - `.ps1` — run via `powershell -ExecutionPolicy Bypass -File` - `.jar` — run via `java -jar` - Shell scripts (`.sh`, `.command`) on macOS / Linux - **Archives** (`.zip`, `.tar`, `.tar.gz`) — automatically extracted, then an entrypoint inside is run (see below) #### Pre-run options All options below can be passed as individual capabilities inside `tb:options`, or set on the advanced object form of `prerun` described further down. | Capability | Object key | Type | Description | | --- | --- | --- | --- | | `prerun` | `executable` | string (URL), object or array | The pre-run file to download and run. A `https://` or `tb://` URL, or an object / array for the advanced forms below. | | `prerun-args` | `args` | string or array | Command-line arguments. As a string, quoted arguments with spaces are preserved, e.g. `--dir "C:\Program Files\App"`. An array is passed through as-is. | | `prerun-wait` | `background` (inverse) | boolean | Wait for the pre-run process to finish before the test starts. Default `false` (fire-and-forget). In the object form use `background: true` to _not_ wait. | | `prerun-timeout` | `timeout` | integer (ms) | Maximum time to wait for the **download** to complete. | | `prerun-wait-timeout` | `waitTimeout` | integer (ms) | When waiting (`prerun-wait`), the maximum time to wait for the pre-run process to **finish**. If it is still running after this, it is terminated. Unset by default (waits indefinitely). | | `prerun-fail-on-error` | `failOnError` | boolean | Fail the session (instead of continuing) when the pre-run cannot run, or — when waiting — exits with a non-zero code or exceeds `prerun-wait-timeout`. Default `false`. | | `prerun-entrypoint` | `entrypoint` | string | For archives: the file inside the archive to run. When omitted we auto-detect `setup.exe`, `install.bat`, `prerun.*`, `run.*`, or the single runnable file at the root. | | `prerun-sha256` | `sha256` | string | Expected SHA-256 checksum of the downloaded file. Verified before execution; a mismatch fails the pre-run. | | `prerun-env` | `env` | JSON object | Extra environment variables for the pre-run process. | | `prerun-cwd` | `cwd` | string | Working directory the pre-run process is started in. | | `prerun-platform` | `platform` | string | Only run on the given platform(s): `windows`, `mac`, `linux` (comma-separated). Useful with the array form. | #### Advanced object form Instead of a plain URL, `prerun` accepts an object so you can set every option in one place: ```javascript "tb:options": { "prerun": { "executable": "https://.../installer.msi", "args": ["/quiet"], "wait": true, "timeout": 120000, "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "failOnError": true } } ``` #### Archives & entrypoint When the download is a `.zip`, `.tar` or `.tar.gz`, we extract it and run an entrypoint inside. Name it explicitly with `entrypoint`, or let us auto-detect it: ```javascript "tb:options": { "prerun": { "executable": "https://.../bundle.zip", "entrypoint": "setup.exe", "args": ["/S"], "wait": true } } ``` #### Inline scripts For small tweaks you don't need to host a file — provide the script inline and pick a shell (`sh`, `bash`, `cmd`, `bat`, `powershell` or `python`). Set `scriptEncoding: "base64"` if you base64-encode the body. ```javascript "tb:options": { "prerun": { "script": "Set-ItemProperty -Path 'HKCU:\\Console' -Name 'QuickEdit' -Value 0", "shell": "powershell", "wait": true } } ``` #### Multiple pre-runs Pass an array to run several pre-runs in order. Each entry can be a URL, an object, or an inline script, and carries its own options: ```javascript "tb:options": { "prerun": [ { "executable": "https://.../install-cert.exe", "args": ["/S"], "wait": true }, { "script": "echo done > C:\\prerun.log", "shell": "cmd", "wait": true } ] } ``` #### Per-platform pre-runs Reuse the same capabilities across a mixed pool by keying the pre-run per operating system. We pick the entry matching the VM's platform, falling back to `default` when present. Keys without a match are skipped (not an error). ```javascript "tb:options": { "prerun": { "windows": { "executable": "https://.../setup.exe", "args": ["/S"] }, "mac": { "script": "defaults write ...", "shell": "sh" }, "default": "https://.../fallback.sh" } } ``` Pre-run failures are non-fatal by default: if the download or script fails, your test still starts. Set `failOnError` (or `prerun-fail-on-error`) to make a failed pre-run stop the session instead. ### Edit hostnames You can specify an array of objects containing the keys `ip` and `domain`. We will write these values to the hosts file of the VM. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["hosts"] = [{ ip: '127.0.0.1', domain: 'mydomain' }] ``` ```java List> hosts = new ArrayList<>(); Map entry = new HashMap<>(); entry.put("ip", "127.0.0.1"); entry.put("domain", "mydomain"); hosts.add(entry); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("hosts", hosts); ``` ```php $caps = array( "hosts" => array(array("ip" => "127.0.0.1", "domain" => "mydomain")) ); ``` ```python capabilities = { "hosts": [{"ip": "127.0.0.1", "domain": "mydomain"}] } ``` ```javascript const capabilities = { "hosts": [{ ip: "127.0.0.1", domain: "mydomain" }] } ``` ```csharp var hosts = new List> { new Dictionary { ["ip"] = "127.0.0.1", ["domain"] = "mydomain" } }; DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("hosts", hosts); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "WIN11", browserName: "chrome", browserVersion: "latest", "tb:options" => { "hosts" : [{ ip: '127.0.0.1', domain: 'mydomain' }] } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); List> hosts = new ArrayList<>(); Map entry = new HashMap<>(); entry.put("ip", "127.0.0.1"); entry.put("domain", "mydomain"); hosts.add(entry); tbOptions.setCapability("hosts", hosts); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "WIN11"); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "chrome"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('WIN11'); $capabilities->setCapability('tb:options', array( 'hosts' => [{ ip: '127.0.0.1', domain: 'mydomain' }] )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'hosts': [{ ip: '127.0.0.1', domain: 'mydomain' }] } chromeOpts = { 'browserName': "chrome", 'platformName': "WIN11", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "platformName": 'WIN11', /** Google requires "w3c" to be set in "goog:chromeOptions" as true if you're using ChromeDriver version 74 or lower. * Based on this commit: https://chromium.googlesource.com/chromium/src/+/2b49880e2481658e0702fd6fe494859bca52b39c * ChromeDriver now uses w3c by default from version 75+ so setting this option will no longer be a requirement **/ "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "hosts": [{ ip: '127.0.0.1', domain: 'mydomain' }] } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["hosts"] = [{ ip: '127.0.0.1', domain: 'mydomain' }] }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` | Value Type | | --- | | array of objects `({ ip: "", domain: "" })` | ### Upload file When you specify an URL (`upload`) and fileName (`uploadFilepath`), we will automatically download the file from the URL and save it in the uploadFilepath. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["upload"] = "https://..." ``` ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("upload", "https://..."); ``` ```php $caps = array( "upload" => "https://..." ); ``` ```python capabilities = { "upload" : "https://..." } ``` ```javascript const capabilities = { "upload" : "https://..." } ``` ```csharp DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("upload", "https://..."); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "WIN11", browserName: "chrome", browserVersion: "latest", "tb:options" => { "upload" : "https://..." } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("upload", "https://..."); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "WIN11"); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "chrome"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('WIN11'); $capabilities->setCapability('tb:options', array( 'upload' => "https://..." )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'upload': "https://..." } chromeOpts = { 'browserName': "chrome", 'platformName': "WIN11", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "platformName": 'WIN11', /** Google requires "w3c" to be set in "goog:chromeOptions" as true if you're using ChromeDriver version 74 or lower. * Based on this commit: https://chromium.googlesource.com/chromium/src/+/2b49880e2481658e0702fd6fe494859bca52b39c * ChromeDriver now uses w3c by default from version 75+ so setting this option will no longer be a requirement **/ "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "upload": "https://..." } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["upload"] = "https://..." }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` | Value Type | | --- | | string (URL) | We recommend using these directories to save your files: For Windows (`C:\test\`), for Linux/macOS: (`/tmp/`). You can also use the home directory tilde to specify the user's directory: `~/Desktop/sample.pdf` Example: ```javascript "uploadFilepath" : "C:\\test\\myfile.ext" ``` Example (Selenium 4): ```javascript "tb:options" : { "uploadFilepath" : "C:\\test\\myfile.ext" } ``` | Value Type | | --- | | string (URL) | ### Upload Multiple Files Specify an array of objects, containing keys `url` and `filePath`. We will automatically download these files and put them in the filePaths you specify. We recommend using these directories to save your files: For Windows (`C:\test\`), for Linux/macOS: (`/tmp/`). You can also use the home directory tilde to specify the user's directory: `~/Desktop/sample.pdf` [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["uploadMultiple"] = [{ url: "", filePath: "" }] ``` ```java List> uploads = new ArrayList<>(); Map file = new HashMap<>(); file.put("url", "https://example.com/file.pdf"); file.put("filePath", "/tmp/file.pdf"); uploads.add(file); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("uploadMultiple", uploads); ``` ```php $caps = array( "uploadMultiple" => array(array("url" => "https://example.com/file.pdf", "filePath" => "/tmp/file.pdf")) ); ``` ```python capabilities = { "uploadMultiple": [{"url": "https://example.com/file.pdf", "filePath": "/tmp/file.pdf"}] } ``` ```javascript const capabilities = { "uploadMultiple": [{ url: "https://example.com/file.pdf", filePath: "/tmp/file.pdf" }] } ``` ```csharp var uploads = new List> { new Dictionary { ["url"] = "https://example.com/file.pdf", ["filePath"] = "/tmp/file.pdf" } }; DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("uploadMultiple", uploads); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "WIN11", browserName: "chrome", browserVersion: "latest", "tb:options" => { "uploadMultiple" : [{ url: "", filePath: "" }] } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); List> uploads = new ArrayList<>(); Map file = new HashMap<>(); file.put("url", "https://example.com/file.pdf"); file.put("filePath", "/tmp/file.pdf"); uploads.add(file); tbOptions.setCapability("uploadMultiple", uploads); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "WIN11"); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "chrome"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('WIN11'); $capabilities->setCapability('tb:options', array( 'uploadMultiple' => array(array("url" => "https://example.com/file.pdf", "filePath" => "/tmp/file.pdf")) )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'uploadMultiple': [{'url': "https://example.com/file.pdf", 'filePath': "/tmp/file.pdf"}] } chromeOpts = { 'browserName': "chrome", 'platformName': "WIN11", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "platformName": 'WIN11', /** Google requires "w3c" to be set in "goog:chromeOptions" as true if you're using ChromeDriver version 74 or lower. * Based on this commit: https://chromium.googlesource.com/chromium/src/+/2b49880e2481658e0702fd6fe494859bca52b39c * ChromeDriver now uses w3c by default from version 75+ so setting this option will no longer be a requirement **/ "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "uploadMultiple": [{ url: "https://example.com/file.pdf", filePath: "/tmp/file.pdf" }] } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["uploadMultiple"] = new List> { new Dictionary { ["url"] = "https://example.com/file.pdf", ["filePath"] = "/tmp/file.pdf" } } }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` | Value Type | | --- | | array of objects, `{ url: "", filePath: "" }` | ### Geolocation Testing We provide an option where you can specify from which country you'd like to run the test from. Once you specify this option, the virtual machine we provision for your test will be configured to use a proxy in the country you specified. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["testingbot.geoCountryCode"] = "DE" ``` ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("testingbot.geoCountryCode", "DE"); ``` ```php $caps = array( "testingbot.geoCountryCode" => "DE" ); ``` ```python capabilities = { "testingbot.geoCountryCode" : "DE" } ``` ```javascript const capabilities = { "testingbot.geoCountryCode" : "DE" } ``` ```csharp DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("testingbot.geoCountryCode", "DE"); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "WIN11", browserName: "chrome", browserVersion: "latest", "tb:options" => { "testingbot.geoCountryCode" : "DE" } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("testingbot.geoCountryCode", "DE"); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "WIN11"); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "chrome"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('WIN11'); $capabilities->setCapability('tb:options', array( 'testingbot.geoCountryCode' => "DE" )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'testingbot.geoCountryCode': "DE" } chromeOpts = { 'browserName': "chrome", 'platformName': "WIN11", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "platformName": 'WIN11', /** Google requires "w3c" to be set in "goog:chromeOptions" as true if you're using ChromeDriver version 74 or lower. * Based on this commit: https://chromium.googlesource.com/chromium/src/+/2b49880e2481658e0702fd6fe494859bca52b39c * ChromeDriver now uses w3c by default from version 75+ so setting this option will no longer be a requirement **/ "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "testingbot.geoCountryCode": "DE" } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["testingbot.geoCountryCode"] = "DE" }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` Specify `testingbot.geoCountryCode` with one of the following country codes: - **'\*'** : this will take a random country from the list below - **'AU'** : Australia - **'BE'** : Belgium - **'BR'** : Brazil - **'CA'** : Canada - **'CL'** : Chile - **'FR'** : France - **'DE'** : Germany - **'IN'** : India - **'IT'** : Italy - **'JP'** : Japan - **'NO'** : Norway - **'SG'** : Singapore - **'ZA'** : South Africa - **'SE'** : Sweden - **'CH'** : Switzerland - **'AE'** : United Arab Emirates - **'GB'** : United Kingdom - **'US'** : United States **Important:** this does not work on Android 4.4 ### Localhost Testing By default, the TestingBot remote VMs are not able to access your localhost, as they are running in a separate network. However, you can use the `localhost` address to access your local web server when using the [TestingBot Tunnel](https://testingbot.com/support/tunnel). Once you have the TestingBot Tunnel running, these localhost ports will be forwarded to your machine: - 80 - 443 - 8080 - 3030 - 3000 - 3001 - 3400 You can also specify additional ports to forward by using the `localHttpPorts` capability. If the localhost port is expecting SSL traffic, you can specify the port in the `localHttpsPorts` capability. [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) [Python](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) ```ruby caps = { "tb:options" => { "localHttpPorts" : [3002, 8081] } } ``` ```java MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("localHttpPorts", [3002, 8081]); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("tb:options", tbOptions); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setCapability('tb:options', array( "localHttpPorts" => [3002, 8081] )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'localHttpPorts': [3002, 8081] } chromeOpts = { 'browserName': "chrome", 'platformName': "WIN11", 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "tb:options": { "localHttpPorts": [3002, 8081] } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["testingbot.localHttpPorts"] = [3002, 8081] }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` ### Change Test Name Add a name to this test, which will show up in our member area and API. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["name"] = "My Test Name" ``` ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("name", "My Test Name"); ``` ```php $caps = array( "name" => "My Test Name" ); ``` ```python capabilities = { "name" : "My Test Name" } ``` ```javascript const capabilities = { "name" : "My Test Name" } ``` ```csharp DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("name", "My Test Name"); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "WIN11", browserName: "chrome", browserVersion: "latest", "tb:options" => { "name" : "My Test Name" } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("name", "My Test Name"); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "WIN11"); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "chrome"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('WIN11'); $capabilities->setCapability('tb:options', array( 'name' => "My Test Name" )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'name': "My Test Name" } chromeOpts = { 'browserName': "chrome", 'platformName': "WIN11", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "platformName": 'WIN11', /** Google requires "w3c" to be set in "goog:chromeOptions" as true if you're using ChromeDriver version 74 or lower. * Based on this commit: https://chromium.googlesource.com/chromium/src/+/2b49880e2481658e0702fd6fe494859bca52b39c * ChromeDriver now uses w3c by default from version 75+ so setting this option will no longer be a requirement **/ "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "name": "My Test Name" } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["name"] = "My Test Name" }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` | Value Type | Default Value | | --- | --- | | string | unnamed test | ### Group Tests A key you can use to group certain tests in the same build (for example in Jenkins). The builds will appear in our member area. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["build"] = "My First Build" ``` ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("build", "My First Build"); ``` ```php $caps = array( "build" => "My First Build" ); ``` ```python capabilities = { "build" : "My First Build" } ``` ```javascript const capabilities = { "build" : "My First Build" } ``` ```csharp DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("build", "My First Build"); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "WIN11", browserName: "chrome", browserVersion: "latest", "tb:options" => { "build" : "My First Build" } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("build", "My First Build"); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "WIN11"); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "chrome"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('WIN11'); $capabilities->setCapability('tb:options', array( 'build' => "My First Build" )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'build': "My First Build" } chromeOpts = { 'browserName': "chrome", 'platformName': "WIN11", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "platformName": 'WIN11', /** Google requires "w3c" to be set in "goog:chromeOptions" as true if you're using ChromeDriver version 74 or lower. * Based on this commit: https://chromium.googlesource.com/chromium/src/+/2b49880e2481658e0702fd6fe494859bca52b39c * ChromeDriver now uses w3c by default from version 75+ so setting this option will no longer be a requirement **/ "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "build": "My First Build" } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["build"] = "My First Build" }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` | Value Type | | --- | | string | ### Idle Timeout The maximum amount of time a browser will wait before proceeding to the next step in your test. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["idletimeout"] = 130 ``` ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("idletimeout", 130); ``` ```php $caps = array( "idletimeout" => 130 ); ``` ```python capabilities = { "idletimeout" : 130 } ``` ```javascript const capabilities = { "idletimeout" : 130 } ``` ```csharp DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("idletimeout", 130); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "WIN11", browserName: "chrome", browserVersion: "latest", "tb:options" => { "idletimeout" : 130 } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("idletimeout", 130); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "WIN11"); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "chrome"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('WIN11'); $capabilities->setCapability('tb:options', array( 'idletimeout' => 130 )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'idletimeout': 130 } chromeOpts = { 'browserName': "chrome", 'platformName': "WIN11", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "platformName": 'WIN11', /** Google requires "w3c" to be set in "goog:chromeOptions" as true if you're using ChromeDriver version 74 or lower. * Based on this commit: https://chromium.googlesource.com/chromium/src/+/2b49880e2481658e0702fd6fe494859bca52b39c * ChromeDriver now uses w3c by default from version 75+ so setting this option will no longer be a requirement **/ "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "idletimeout": 130 } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["idletimeout"] = 130 }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` | Value Type | Default Value | | --- | --- | | int (specify number of seconds) | 130 seconds | ### Maximum Test Duration The maximum duration for a single test. This is a safeguard to prevent bad tests from using up your credits. We generally recommend to keep tests short (less than 10 minutes). It's better to split up large tests in smaller individual tests. This keeps your tests fast and allows for more parallelization of your tests. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["maxduration"] = 1800 ``` ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("maxduration", 1800); ``` ```php $caps = array( "maxduration" => 1800 ); ``` ```python capabilities = { "maxduration" : 1800 } ``` ```javascript const capabilities = { "maxduration" : 1800 } ``` ```csharp DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("maxduration", 1800); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "WIN11", browserName: "chrome", browserVersion: "latest", "tb:options" => { "maxduration" : 1800 } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("maxduration", 1800); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "WIN11"); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "chrome"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('WIN11'); $capabilities->setCapability('tb:options', array( 'maxduration' => 1800 )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'maxduration': 1800 } chromeOpts = { 'browserName': "chrome", 'platformName': "WIN11", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "platformName": 'WIN11', /** Google requires "w3c" to be set in "goog:chromeOptions" as true if you're using ChromeDriver version 74 or lower. * Based on this commit: https://chromium.googlesource.com/chromium/src/+/2b49880e2481658e0702fd6fe494859bca52b39c * ChromeDriver now uses w3c by default from version 75+ so setting this option will no longer be a requirement **/ "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "maxduration": 1800 } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["maxduration"] = 1800 }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` | Value Type | Default Value | | --- | --- | | int (specify number of seconds) | 1800 seconds (30 minutes) | ### Custom Metadata Send along custom data, for example your release, server, commit hash, ... This will show up on the test detail page in the TestingBot member area. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["extra"] = "Extra Information" ``` ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("extra", "Extra Information"); ``` ```php $caps = array( "extra" => "Extra Information" ); ``` ```python capabilities = { "extra" : "Extra Information" } ``` ```javascript const capabilities = { "extra" : "Extra Information" } ``` ```csharp DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("extra", "Extra Information"); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "WIN11", browserName: "chrome", browserVersion: "latest", "tb:options" => { "extra" : "Extra Information" } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("extra", "Extra Information"); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "WIN11"); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "chrome"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('WIN11'); $capabilities->setCapability('tb:options', array( 'extra' => "Extra Information" )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'extra': "Extra Information" } chromeOpts = { 'browserName': "chrome", 'platformName': "WIN11", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "platformName": 'WIN11', /** Google requires "w3c" to be set in "goog:chromeOptions" as true if you're using ChromeDriver version 74 or lower. * Based on this commit: https://chromium.googlesource.com/chromium/src/+/2b49880e2481658e0702fd6fe494859bca52b39c * ChromeDriver now uses w3c by default from version 75+ so setting this option will no longer be a requirement **/ "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "extra": "Extra Information" } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["extra"] = "Extra Information" }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` | Value Type | | --- | | string | ### Load custom browser extensions If you specify this desired capability with the URL to your Chrome extension's `.crx` file, Safari App/Web extension file, or Firefox addon's `.xpi` file, we will download the extension, which will be added to the browser before your test starts. More information is available in the [automated browser extension testing documentation](https://testingbot.com/support/web-automate/selenium/browser-extension). [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["load-extension"] = "https://..." ``` ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("load-extension", "https://..."); ``` ```php $caps = array( "load-extension" => "https://..." ); ``` ```python capabilities = { "load-extension" : "https://..." } ``` ```javascript const capabilities = { "load-extension" : "https://..." } ``` ```csharp DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("load-extension", "https://..."); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "WIN11", browserName: "chrome", browserVersion: "latest", "tb:options" => { "load-extension" : "https://..." } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("load-extension", "https://..."); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "WIN11"); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "chrome"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('WIN11'); $capabilities->setCapability('tb:options', array( 'load-extension' => "https://..." )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'load-extension': "https://..." } chromeOpts = { 'browserName': "chrome", 'platformName': "WIN11", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "platformName": 'WIN11', /** Google requires "w3c" to be set in "goog:chromeOptions" as true if you're using ChromeDriver version 74 or lower. * Based on this commit: https://chromium.googlesource.com/chromium/src/+/2b49880e2481658e0702fd6fe494859bca52b39c * ChromeDriver now uses w3c by default from version 75+ so setting this option will no longer be a requirement **/ "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "load-extension": "https://..." } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["load-extension"] = "https://..." }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` | Browser | Supported Format | Example | | --- | --- | --- | | Chrome | `.zip` file, containing the extension source code (`manifest.json`, `src/ directory`, ...) | `https://.../extension.zip` | | Firefox | `.zip` file, containing the extension source code (`manifest.json`, `src/ directory`, `lib/ directory`, ...) | `https://.../extension.zip` | | Edge | `.zip` file, containing the extension source code (`manifest.json`, `src/ directory`, ...) | `https://.../extension.zip` | | Safari (and Safari Technology Preview) | `Safari Web Extension, Safari App Extension or .safariextz` file | `https://.../extension.zip or https://.../extension.safariextz` | With [TestingBot Storage](https://testingbot.com/support/api/storage#upload), you can upload your executable on our servers. The advantage of this is that our test VMs can immediately download your executable from our own network, which is much faster than downloading from the public internet. Once the file is uploaded via TestingBot Storage, you can use `"prerun" : "tb://..."` ### Grouping Specify in which groups you want to see the test results. You can group results to have an easy overview of tests across projects. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["groups"] = "group1,group2" ``` ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("groups", "group1,group2"); ``` ```php $caps = array( "groups" => "group1,group2" ); ``` ```python capabilities = { "groups" : "group1,group2" } ``` ```javascript const capabilities = { "groups" : "group1,group2" } ``` ```csharp DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("groups", "group1,group2"); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "WIN11", browserName: "chrome", browserVersion: "latest", "tb:options" => { "groups" : "group1,group2" } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("groups", "group1,group2"); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "WIN11"); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "chrome"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('WIN11'); $capabilities->setCapability('tb:options', array( 'groups' => "group1,group2" )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'groups': "group1,group2" } chromeOpts = { 'browserName': "chrome", 'platformName': "WIN11", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "platformName": 'WIN11', /** Google requires "w3c" to be set in "goog:chromeOptions" as true if you're using ChromeDriver version 74 or lower. * Based on this commit: https://chromium.googlesource.com/chromium/src/+/2b49880e2481658e0702fd6fe494859bca52b39c * ChromeDriver now uses w3c by default from version 75+ so setting this option will no longer be a requirement **/ "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "groups": "group1,group2" } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["groups"] = "group1,group2" }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` | Value Type | | --- | | string (comma-separated) | ### AutoClicker Specify one or more of the possible options below to have TestingBot automatically click certain dialogs which are unable to be automated via Selenium WebDriver. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["autoclick"] = [] ``` ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("autoclick", []); ``` ```php $caps = array( "autoclick" => [] ); ``` ```python capabilities = { "autoclick" : [] } ``` ```javascript const capabilities = { "autoclick" : [] } ``` ```csharp DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("autoclick", []); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "WIN11", browserName: "chrome", browserVersion: "latest", "tb:options" => { "autoclick" : [] } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("autoclick", []); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "WIN11"); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "chrome"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('WIN11'); $capabilities->setCapability('tb:options', array( 'autoclick' => [] )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'autoclick': [] } chromeOpts = { 'browserName': "chrome", 'platformName': "WIN11", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "platformName": 'WIN11', /** Google requires "w3c" to be set in "goog:chromeOptions" as true if you're using ChromeDriver version 74 or lower. * Based on this commit: https://chromium.googlesource.com/chromium/src/+/2b49880e2481658e0702fd6fe494859bca52b39c * ChromeDriver now uses w3c by default from version 75+ so setting this option will no longer be a requirement **/ "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "autoclick": [] } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["autoclick"] = [] }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` | Value Type | Possible Values: | | --- | --- | | array of options | `chrome_extension_add`, `chrome_extension_permissions_allow` | ### Sikuli Specify an URL to a zip-file containing one or more `.sikulu` projects. These will automatically start before your test runs, so that you can do certain automated tasks which Selenium cannot do (clicking native dialogs, ...) Find more information regarding [Sikuli Cloud Testing](https://testingbot.com/support/other/sikuli). [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = Selenium::WebDriver::Remote::Capabilities.new caps["sikuli"] = "https://..." ``` ```java DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("sikuli", "https://..."); ``` ```php $caps = array( "sikuli" => "https://..." ); ``` ```python capabilities = { "sikuli" : "https://..." } ``` ```javascript const capabilities = { "sikuli" : "https://..." } ``` ```csharp DesiredCapabilities caps = new DesiredCapabilities(); caps.SetCapability("sikuli", "https://..."); ``` **Selenium W3C Example:** [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps = { platformName: "WIN11", browserName: "chrome", browserVersion: "latest", "tb:options" => { "sikuli" : [] } } ``` ```java ChromeOptions chromeOpts = new ChromeOptions(); chromeOpts.setExperimentalOption("w3c", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("sikuli", []); DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts); caps.setCapability("platformName", "WIN11"); caps.setCapability("tb:options", tbOptions); caps.setCapability("browserName", "chrome"); ``` ```php $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setPlatform('WIN11'); $capabilities->setCapability('tb:options', array( 'sikuli' => [] )); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); ``` ```python tbOptions = { 'sikuli': [] } chromeOpts = { 'browserName': "chrome", 'platformName': "WIN11", 'goog:chromeOptions': {'w3c': True}, 'tb:options': tbOptions } self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "platformName": 'WIN11', /** Google requires "w3c" to be set in "goog:chromeOptions" as true if you're using ChromeDriver version 74 or lower. * Based on this commit: https://chromium.googlesource.com/chromium/src/+/2b49880e2481658e0702fd6fe494859bca52b39c * ChromeDriver now uses w3c by default from version 75+ so setting this option will no longer be a requirement **/ "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "sikuli": [] } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "WIN11", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["sikuli"] = [] }; chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` | Value Type | | --- | | string (URL to zipped Sikuli project) | ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/network # Custom WebDriver Commands by TestingBot TestingBot has added custom WebDriver commands for Chrome and Microsoft Edge browsers. With these custom commands, you can **retrieve various metrics** , **throttle network speed** and **monitor/adjust network requests** during your test. These custom commands can be used with WebDriver's `JavascriptExecutor`. **Important:** to be able to use these commands, **you need to pass** `debugging: true` in your desired capabilities. ## Throttle Network ### Description: This call allows you to throttle network speed, or even simulate no network connection (offline-mode). [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java ((JavascriptExecutor) driver).executeScript("tb:throttle", "3G"); // or Map throttleMap = new HashMap<>(); throttleMap.put("downloadSpeed", 10 * 1024); throttleMap.put("uploadSpeed", 10 * 1024); throttleMap.put("latency", 0); throttleMap.put("loss", 0); ((JavascriptExecutor) driver).executeScript("tb:throttle", throttleMap); ``` ```python driver.execute_script("tb:throttle", "3G") # or driver.execute_script("tb:throttle", { "downloadSpeed": 10 * 1024, "uploadSpeed": 10 * 1024, "latency": 0, "loss": 0 }) ``` ```javascript await driver.executeScript('tb:throttle', '3G'); // or await driver.executeScript('tb:throttle', { "downloadSpeed": 10 * 1024, "uploadSpeed": 10 * 1024, "latency": 0, "loss": 0 }); ``` ```ruby driver.execute_script("tb:throttle", "3G") # or driver.execute_script("tb:throttle", { "downloadSpeed" => 10 * 1024, "uploadSpeed" => 10 * 1024, "latency" => 0, "loss" => 0 }) ``` ```php $driver->executeScript("tb:throttle", ["3G"]); // or $driver->executeScript("tb:throttle", [ "downloadSpeed" => 10 * 1024, "uploadSpeed" => 10 * 1024, "latency" => 0, "loss" => 0 ]); ``` ```csharp ((IJavaScriptExecutor)driver).ExecuteScript("tb:throttle", "3G"); // or ((IJavaScriptExecutor)driver).ExecuteScript("tb:throttle", new Dictionary { ["downloadSpeed"] = 10240, ["uploadSpeed"] = 10240, ["latency"] = 0, ["loss"] = 0 }); ``` #### Options: | `tb:throttle` Specifying a predefined condition | Condition \| Download Speed (kb/sec) \| Upload Speed (kb/sec) \| Latency (ms) \|\| `airplane` Simulate Offline Mode \| 0 \| 0 \| 0 \| \| `disable` Default - useful if you want to undo the `offline` command. \| no throttle \| no throttle \| no throttle \| \| `edge` \| 250 \| 150 \| 300 \| \| `3G` \| 400 \| 100 \| 100 \| \| `4G` \| 18000 \| 9000 \| 100 \| | | `tb:throttle` Specify custom settings | `downloadSpeed` \| `uploadSpeed` \| `latency` \|\| ... kb / sec \| ... kb / sec \| ... ms \| | ## Intercept/Mock ### Description: With these commands, you can intercept and even fake responses during your test. Blacklist requests, add/modify headers, mock responses, ... The `url` parameter supports wildcard statements, for example: `https://www.testingbot.com/*`. ### Redirect a URL Instructs Chrome/Edge to redirect to `redirect` upon navigating to `url`. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java Map interceptMap = new HashMap<>(); interceptMap.put("redirect", "https://www.google.com"); interceptMap.put("url", "https://testingbot.com"); ((JavascriptExecutor) driver).executeScript("tb:intercept", interceptMap); ``` ```python driver.execute_script("tb:intercept", { "redirect": "https://google.com", "url": "https://testingbot.com" }) ``` ```javascript await driver.executeScript('tb:intercept', { "redirect": "https://google.com", "url": "https://testingbot.com" }); ``` ```ruby driver.execute_script("tb:intercept", { "redirect" => "https://www.google.com", "url" => "https://testingbot.com" }) ``` ```php $driver->executeScript("tb:intercept", [ "redirect" => "https://google.com", "url" => "https://testingbot.com" ]); ``` ```csharp ((IJavaScriptExecutor)driver).ExecuteScript("tb:intercept", new Dictionary { ["redirect"] = "https://www.google.com", ["url"] = "https://testingbot.com" }); ``` ### Trigger Failure Allows you to return a specific error for a specific request. [Possible errors](https://chromedevtools.github.io/devtools-protocol/tot/Network#type-ErrorReason): - Failed - Aborted - TimedOut - AccessDenied - ConnectionClosed - ConnectionReset - ConnectionRefused - ConnectionAborted - ConnectionFailed - NameNotResolved - InternetDisconnected - AddressUnreachable - BlockedByClient - BlockedByResponse [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java Map interceptMap = new HashMap<>(); interceptMap.put("error", "ConnectionRefused"); interceptMap.put("url", "https://testingbot.com"); ((JavascriptExecutor) driver).executeScript("tb:intercept", interceptMap); ``` ```python driver.execute_script("tb:intercept", { "error": "ConnectionRefused", "url": "https://testingbot.com" }) ``` ```javascript await driver.executeScript('tb:intercept', { "error": "ConnectionRefused", "url": "https://testingbot.com" }); ``` ```ruby driver.execute_script("tb:intercept", { "error" => "ConnectionRefused", "url" => "https://testingbot.com" }) ``` ```php $driver->executeScript("tb:intercept", [ "error" => "ConnectionRefused", "url" => "https://testingbot.com" ]); ``` ```csharp ((IJavaScriptExecutor)driver).ExecuteScript("tb:intercept", new Dictionary { ["error"] = "ConnectionRefused", ["url"] = "https://testingbot.com" }); ``` ### Mock Response Allows you to return a different response (body or headers). [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java Map mockMap = new HashMap<>(); Map headersMap = new HashMap<>(); headersMap.put("x-a-header", "Sample"); Map responseMap = new HashMap<>(); responseMap.put("statusCode", 200); responseMap.put("headers", headersMap); responseMap.put("body", "Hello World"); mockMap.put("response", responseMap); mockMap.put("url", "https://testingbot.com"); ((JavascriptExecutor) driver).executeScript("tb:intercept", mockMap); ``` ```python driver.execute_script("tb:intercept", { "response": { "statusCode": 200, "headers": { "x-a-header": "Sample" }, "body": "Hello World" }, "url": "https://testingbot.com" }) ``` ```javascript await driver.executeScript('tb:intercept', { "response": { "statusCode": 200, "headers": { "x-a-header": "Sample" }, "body": "Hello World" }, "url": "https://testingbot.com" }); ``` ```ruby driver.execute_script("tb:intercept", { "response" => { "statusCode" => 200, "headers" => { "x-a-header" => "Sample" }, "body" => "Hello World" }, "url" => "https://testingbot.com" }) ``` ```php $driver->executeScript("tb:intercept", [ "response" => [ "statusCode" => 200, "headers" => [ "x-a-header" => "Sample" ], "body" => "Hello World" ], "url" => "https://testingbot.com" ]); ``` ```csharp ((IJavaScriptExecutor)driver).ExecuteScript("tb:intercept", new Dictionary { ["response"] = new Dictionary { ["statusCode"] = 200, ["headers"] = new Dictionary { ["x-a-header"] = "Sample" }, ["body"] = "Hello World" }, ["url"] = "https://testingbot.com" }); ``` ## Retrieve Network Log ### Description: With the `tb:network` command you can fetch a log of network calls from Chrome and Microsoft Edge. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java ((JavascriptExecutor) driver).executeScript("tb:network"); ``` ```python driver.execute_script("tb:network") ``` ```javascript await driver.executeScript('tb:network'); ``` ```ruby driver.execute_script("tb:network") ``` ```php $driver->executeScript("tb:network"); ``` ```csharp ((IJavaScriptExecutor)driver).ExecuteScript("tb:network"); ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/php ### PHP Examples: - [Behat](https://testingbot.com/support/web-automate/selenium/php/behat-mink) - [Codeception](https://testingbot.com/support/web-automate/selenium/php/codeception) - [Laravel Dusk](https://testingbot.com/support/web-automate/selenium/php/laravel-dusk) - [PHPUnit](https://testingbot.com/support/web-automate/selenium/php/phpunit) - [SimpleTest](https://testingbot.com/support/web-automate/selenium/php/simpletest) # PHP Automated Testing Let's start with making sure PHP is available on your system. **For Windows:** - You can download PHP for Windows from [PHP For Windows](https://windows.php.net/download/). - Run the installer and follow the setup wizard to install PHP. **For Linux:** ```bash sudo apt-get install curl libcurl4 libcurl4-openssl-dev php ``` **For macOS:** ```bash brew install php ``` [![Video tutorial on How to run PHP Automated Test on TestingBot](https://testingbot.com/assets/videos/php-automated-4985ae1c28644d70de6288090302df66c8c7c02227aed461ca24c0659572f688.webp)](https://testingbot.com#) about:blank See our [PHP example repository](https://github.com/testingbot/php-phpunit-example) for a first example on how to run PHP tests in parallel on TestingBot. ## Installation With TestingBot you can easily run your automated tests with [any PHP test framework](https://testingbot.com#frameworks), here's a first example with **php-webdriver** : ```bash php composer.phar require php-webdriver/webdriver ``` Save the example code below in a file called `sampletest.php` ```php setCapability('platformName', 'WIN11'); $caps->setCapability('browserVersion', 'latest'); $caps->setCapability('tb:options', [ 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'First Test' ]); $driver = RemoteWebDriver::create( "https://hub.testingbot.com/wd/hub", $caps, 120000 ); $driver->get("https://www.google.com"); $element = $driver->findElement(WebDriverBy::name("q")); if ($element) { $element->sendKeys("TestingBot"); $element->submit(); } print $driver->getTitle(); $driver->quit(); ?> ``` You can now run this test: ```bash php sampletest.php ``` ### Configuring capabilities To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before:** ```php $caps = DesiredCapabilities::chrome(); $driver = RemoteWebDriver::create("http://localhost:4444/wd/hub", $caps, 120000); ``` **After:** ```php $caps = DesiredCapabilities::chrome(); $caps->setCapability('platformName', 'WIN11'); $caps->setCapability('browserVersion', 'latest'); $caps->setCapability('tb:options', [ 'key' => 'API_KEY', 'secret' => 'API_SECRET' ]); $driver = RemoteWebDriver::create("https://hub.testingbot.com/wd/hub", $caps, 120000); ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ```php $driver = RemoteWebDriver::create("https://hub.testingbot.com/wd/hub", $caps, 120000); ``` To see how to do this, please select a combination of browser, version and platform in the drop-down menus below. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a PHP WebDriver test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```php setCapability('platformName', 'WIN11'); $caps->setCapability('browserVersion', 'latest'); $caps->setCapability('tb:options', [ 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'Tunnel Test' ]); // Point to the tunnel running on localhost:4445 $driver = RemoteWebDriver::create( "http://localhost:4445/wd/hub", $caps, 120000 ); $driver->get("https://www.google.com"); $element = $driver->findElement(WebDriverBy::name("q")); if ($element) { $element->sendKeys("TestingBot"); $element->submit(); } print $driver->getTitle(); $driver->quit(); ?> ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. To run tests in parallel, we recommend using [Paratest](https://github.com/brianium/paratest), which makes it very easy to run multiple PHPUnit tests simultaneously. ```bash vendor/bin/paratest -p 8 -f --phpunit=vendor/bin/phpunit WebdriverTest.php ``` The command above will run your tests in 8 separate processes (8 tests at once). ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. You can use our [PHP API client](https://github.com/testingbot/testingbot-php) to report back test results. ```php $api = new TestingBot\TestingBotAPI("API_KEY", "API_SECRET"); $api->updateJob($driver->getSessionID(), ['name' => 'mytest', 'success' => true]); ``` ## Other PHP Framework examples - [Behat](https://testingbot.com/support/web-automate/selenium/php/behat-mink) Behat is a BDD framework which runs on PHP. Mink is used for its browser emulation and works nicely together with Behat. - [Codeception](https://testingbot.com/support/web-automate/selenium/php/codeception) Codeception is a BDD-styled PHP testing framework. This testing framework offers good Selenium support. - [Laravel Dusk](https://testingbot.com/support/web-automate/selenium/php/laravel-dusk) Laravel Dusk provides an easy-to-use browser automation testing framework. - [PHPUnit](https://testingbot.com/support/web-automate/selenium/php/phpunit) PHPUnit is the most popular unit testing framework for PHP. It comes with good Selenium WebDriver support and is easy to set up. - [SimpleTest](https://testingbot.com/support/web-automate/selenium/php/simpletest) SimpleTest is a framework for unit testing, web site testing and mock objects for PHP. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/php/behat-mink # Behat Automated Testing See our [Behat 2 example](https://github.com/testingbot/behat-selenium-sample) or [Behat 3 example](https://github.com/testingbot/behat-selenium-sample/tree/behat3) for some simple examples on how to run Behat tests on TestingBot. TestingBot supports Selenium tests using Behat. It's very easy to create or convert Behat tests to use our Selenium Grid. Behat is a Behavior-Driven Development framework for PHP. You can find more information on the [Behat Documentation Pages](https://docs.behat.org/en/latest/). ## Installation Make sure you have `Composer` installed: ```bash curl https://getcomposer.org/installer | php ``` First make sure you've installed Behat. See this example with PHP Composer: ```bash php composer.phar install ``` Next, let's create the necessary files: **composer.json** : ```json { "require": { "behat/behat": "^3.0", "php-webdriver/webdriver": "^1.14", "symfony/yaml": "*" }, "config": { "bin-dir": "bin/" }, "scripts": { "test": "composer single && composer parallel", "single": "./vendor/behat/behat/bin/behat --config=config/single.conf.yml", "parallel": "CONFIG_FILE=config/parallel.conf.yml /usr/bin/env php lib/parallel.php" }, "autoload": { "classmap": ["lib/"] } } ``` Now we can create a simple testcase, written in Behat: **features/single/single.feature** ```yaml Feature: Google Search Functionality Scenario: Can find search results Given I am on "https://www.google.com/ncr" When I search for "Google" Then I should see "Google" ``` **features/bootstrap/FeatureContext.php** : ```php setCapability('tb:options', $config['capabilities'] ?? []); self::$driver = RemoteWebDriver::create($url, $caps); } /** @AfterFeature */ public static function teardownBrowser(AfterFeatureScope $scope) { if (self::$driver) { self::$driver->quit(); } } /** @Given /^I am on "([^"]*)"$/ */ public function iAmOnSite($url) { self::$driver->get($url); } /** @When /^I search for "([^"]*)"$/ */ public function iSearchFor($searchText) { $element = self::$driver->findElement(WebDriverBy::name('q')); $element->sendKeys($searchText); $element->submit(); sleep(5); } /** @Then /^I should see "([^"]*)"$/ */ public function iShouldSee($string) { $source = self::$driver->getPageSource(); if (strpos($source, $string) === false) { throw new Exception("Expected to see: '" . $string . "', actual page did not contain it."); } } } ``` Note: this is a **Behat 3** example. `Behat\Behat\Context\Context` is the Behat 3 interface, the older `BehatContext` base class only existed in Behat 2 and won't resolve under the `behat/behat ^3.0` requirement above. To actually run this test, we first need to add some TestingBot settings to `config/single.conf.yml` ```yaml default: suites: default: paths: - '%paths.base%/features/single' contexts: - FeatureContext: testingbot: server: "hub.testingbot.com" user: "key" secret: "secret" capabilities: build: "behat-selenium-sample" name: "single-behat-test" environments: - browserName: chrome browserVersion: latest platformName: WIN11 ``` You can now run this test on TestingBot using the following command: ```bash php composer.phar single ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a PHP WebDriver test with our Tunnel: 1. [Download TestingBot Tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back. Update `config/single.conf.yml` so the `server` points at the tunnel: ```yaml default: suites: default: paths: - '%paths.base%/features/single' contexts: - FeatureContext: testingbot: server: "localhost:4445" user: "key" secret: "secret" capabilities: name: "behat-tunnel-test" environments: - browserName: chrome browserVersion: latest platformName: WIN11 ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Other PHP Framework examples - [Behat](https://testingbot.com/support/web-automate/selenium/php/behat-mink) Behat is a BDD framework which runs on PHP. Mink is used for its browser emulation and works nicely together with Behat. - [Codeception](https://testingbot.com/support/web-automate/selenium/php/codeception) Codeception is a BDD-styled PHP testing framework. This testing framework offers good Selenium support. - [Laravel Dusk](https://testingbot.com/support/web-automate/selenium/php/laravel-dusk) Laravel Dusk provides an easy-to-use browser automation testing framework. - [PHPUnit](https://testingbot.com/support/web-automate/selenium/php/phpunit) PHPUnit is the most popular unit testing framework for PHP. It comes with good Selenium WebDriver support and is easy to set up. - [SimpleTest](https://testingbot.com/support/web-automate/selenium/php/simpletest) SimpleTest is a framework for unit testing, web site testing and mock objects for PHP. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/php/codeception # Codeception Automated Testing See our [CodeCeption example repository](https://github.com/testingbot/php-codeception-example) for a simple example on how to run CodeCeption tests in parallel on TestingBot. We start with a simple test: ```bash php codecept.phar generate:cept acceptance Welcome ``` Make sure you set these values in `tests/acceptance.suite.yml` ```yaml actor: AcceptanceTester env: chrome: modules: enabled: - WebDriver config: WebDriver: url: 'https://www.google.com' host: 'hub.testingbot.com' port: 443 protocol: 'https' path: '/wd/hub' browser: chrome capabilities: 'browserVersion': 'latest' 'platformName': 'WIN11' 'tb:options': 'key': 'key' 'secret': 'secret' 'name': 'Sample Codeception Tests' firefox: modules: enabled: - WebDriver config: WebDriver: url: 'https://www.google.com' host: 'hub.testingbot.com' port: 443 protocol: 'https' path: '/wd/hub' browser: firefox capabilities: 'browserVersion': 'latest' 'platformName': 'WIN11' 'tb:options': 'key': 'key' 'secret': 'secret' 'name': 'Sample Codeception Tests' ``` Now we can run a basic example where we search for TestingBot on Google. Use a [Cest class](https://codeception.com/docs/AcceptanceTests) (the legacy Cept style with `new AcceptanceTester($scenario)` was removed in modern Codeception): ```php wantTo('Search TestingBot on Google'); $I->amOnPage('/'); $I->fillField('q', 'TestingBot'); $I->pressKey('q', \Facebook\WebDriver\WebDriverKeys::ENTER); $I->see('TestingBot'); } } ``` Additional settings can be found on the [Codeception WebDriver Module page](https://codeception.com/docs/modules/WebDriver). ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. To see how to do this, please select a combination of browser, version and platform in the drop-down menus below. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a Codeception test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back. Edit the configuration in `tests/acceptance.suite.yml`: ```yaml actor: AcceptanceTester env: chrome: modules: enabled: - WebDriver config: WebDriver: url: 'https://www.google.com' host: 'localhost' port: 4445 protocol: 'http' path: '/wd/hub' browser: chrome capabilities: 'browserVersion': 'latest' 'platformName': 'WIN11' 'tb:options': 'name': 'Sample Codeception Tests' ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. Please see [CodeCeption Parallel Execution Docs](https://codeception.com/docs/ParallelExecution) for more information on how to run tests in parallel with [Robo-paracept](https://github.com/Codeception/robo-paracept). ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. We've created an extension for Codeception: [testingbot/codeception-extension](https://github.com/testingbot/codeception-extension) By including this in your tests, Codeception will automatically send back test-meta data to TestingBot (test name, success/failure, ...) ## Other PHP Framework examples - [Behat](https://testingbot.com/support/web-automate/selenium/php/behat-mink) Behat is a BDD framework which runs on PHP. Mink is used for its browser emulation and works nicely together with Behat. - [Codeception](https://testingbot.com/support/web-automate/selenium/php/codeception) Codeception is a BDD-styled PHP testing framework. This testing framework offers good Selenium support. - [Laravel Dusk](https://testingbot.com/support/web-automate/selenium/php/laravel-dusk) Laravel Dusk provides an easy-to-use browser automation testing framework. - [PHPUnit](https://testingbot.com/support/web-automate/selenium/php/phpunit) PHPUnit is the most popular unit testing framework for PHP. It comes with good Selenium WebDriver support and is easy to set up. - [SimpleTest](https://testingbot.com/support/web-automate/selenium/php/simpletest) SimpleTest is a framework for unit testing, web site testing and mock objects for PHP. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/php/laravel-dusk # Laravel Automated Testing Let's start with making sure PHP is available on your system. **For Windows:** - You can download PHP for Windows from [PHP For Windows](https://windows.php.net/download/). - Run the installer and follow the setup wizard to install PHP. **For Linux:** ```bash sudo apt-get install curl libcurl3 libcurl3-dev php ``` **For macOS:** ```bash brew install php ``` ## Installation - Make sure you have [Composer installed](https://getcomposer.org/). - Install the [Laravel Dusk Framework](https://laravel.com/docs/master/dusk) and other dependencies creating this `composer.json` file: - Install the dependencies by running: - Install the necessary Dusk files: ## Configuration If you followed the steps above, you should now have a `tests/DuskTestCase.php` file. You can now generate a sample test file, for example with: ```bash php artisan dusk:make LoginTest ``` To connect your test with TestingBot, you'll need to edit the `tests/DuskTestCase.php` file. Add the imports below to the top of the file and replace the stock `driver()` method: ```php use Facebook\WebDriver\Remote\DesiredCapabilities; use Facebook\WebDriver\Remote\RemoteWebDriver; protected function driver() { $capabilities = DesiredCapabilities::chrome(); $capabilities->setCapability('platformName', 'WIN11'); $capabilities->setCapability('browserVersion', 'latest'); $capabilities->setCapability('tb:options', [ 'key' => env('TESTINGBOT_KEY'), 'secret' => env('TESTINGBOT_SECRET'), 'name' => $this->name() ]); return RemoteWebDriver::create( 'https://hub.testingbot.com/wd/hub', $capabilities, 120000 ); } ``` To run the test, please enter this command: ```bash php artisan dusk ``` ### Configuring capabilities To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before:** ```php $capabilities = DesiredCapabilities::chrome(); $web_driver = RemoteWebDriver::create( 'http://localhost:4444/wd/hub', $capabilities ); ``` **After:** ```php $capabilities = DesiredCapabilities::chrome(); $capabilities->setCapability('platformName', 'WIN11'); $capabilities->setCapability('browserVersion', 'latest'); $capabilities->setCapability('tb:options', [ 'key' => env('TESTINGBOT_KEY'), 'secret' => env('TESTINGBOT_SECRET') ]); $web_driver = RemoteWebDriver::create( 'https://hub.testingbot.com/wd/hub', $capabilities ); ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ```php $capabilities->setCapability('tb:options', [ 'key' => env('TESTINGBOT_KEY'), 'secret' => env('TESTINGBOT_SECRET') ]); $web_driver = RemoteWebDriver::create( 'https://hub.testingbot.com/wd/hub', $capabilities ); ``` To see how to do this, please select a combination of browser, version and platform in the drop-down menus below. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a Laravel Dusk test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```php protected function driver() { $capabilities = DesiredCapabilities::firefox(); $capabilities->setCapability('platformName', 'WIN11'); $capabilities->setCapability('browserVersion', 'latest'); $capabilities->setCapability('tb:options', [ 'key' => env('TESTINGBOT_KEY'), 'secret' => env('TESTINGBOT_SECRET') ]); return RemoteWebDriver::create( 'http://localhost:4445/wd/hub', $capabilities, 120000 ); } ``` ## Other PHP Framework examples - [Behat](https://testingbot.com/support/web-automate/selenium/php/behat-mink) Behat is a BDD framework which runs on PHP. Mink is used for its browser emulation and works nicely together with Behat. - [Codeception](https://testingbot.com/support/web-automate/selenium/php/codeception) Codeception is a BDD-styled PHP testing framework. This testing framework offers good Selenium support. - [PHPUnit](https://testingbot.com/support/web-automate/selenium/php/phpunit) PHPUnit is the most popular unit testing framework for PHP. It comes with good Selenium WebDriver support and is easy to set up. - [SimpleTest](https://testingbot.com/support/web-automate/selenium/php/simpletest) SimpleTest is a framework for unit testing, web site testing and mock objects for PHP. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/php/phpunit # PHPUnit Automated Testing See our [PHPUnit example repository](https://github.com/testingbot/php-phpunit-example) for a simple example on how to run PHPUnit tests in parallel on TestingBot. Let's start with making sure PHP is available on your system. **For Windows:** - You can download PHP for Windows from [PHP For Windows](https://windows.php.net/download/). - Run the installer and follow the setup wizard to install PHP. **For Linux:** ```bash sudo apt-get install curl libcurl4 libcurl4-openssl-dev php ``` **For macOS:** ```bash brew install php ``` ## Installation - Make sure you have [Composer installed](https://getcomposer.org/). - Install PHPUnit and the WebDriver client by running these commands in your terminal: ## PHPUnit Example Paste this example code in a file called `WebDriverTest.php` ```php setCapability('platformName', 'WIN11'); $caps->setCapability('browserVersion', 'latest'); $caps->setCapability('tb:options', [ 'key' => $this->apiKey, 'secret' => $this->apiSecret, 'name' => $this->name() ]); $this->driver = RemoteWebDriver::create( 'https://hub.testingbot.com/wd/hub', $caps, 120000 ); } /** * Use a passed-and-not-errored test hook so we can know whether the test * succeeded from inside tearDown(). PHPUnit 10+ removed $this->hasFailed() * and $this->getName(), so we capture status with @after instead. */ protected function assertPostConditions(): void { parent::assertPostConditions(); $this->testPassed = true; } protected function tearDown(): void { $this->sendTestStatusToTestingBot(); $this->driver->quit(); parent::tearDown(); } public function testTitle(): void { $this->driver->get('https://www.google.com/'); $this->assertEquals('Google', $this->driver->getTitle()); } protected function sendTestStatusToTestingBot(): void { $sessionID = $this->driver->getSessionID(); $postData = http_build_query([ 'test[success]' => $this->testPassed ? 1 : 0, 'test[name]' => $this->name() ]); $curl = curl_init(); curl_setopt($curl, CURLOPT_URL, "https://api.testingbot.com/v1/tests/{$sessionID}"); curl_setopt($curl, CURLOPT_CUSTOMREQUEST, "PUT"); curl_setopt($curl, CURLOPT_RETURNTRANSFER, true); curl_setopt($curl, CURLOPT_USERPWD, "{$this->apiKey}:{$this->apiSecret}"); curl_setopt($curl, CURLOPT_POSTFIELDS, $postData); curl_exec($curl); curl_close($curl); } } ``` To run the test, please enter this command: ```bash ./vendor/bin/phpunit WebDriverTest.php ``` ### Configuring capabilities To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before:** ```php $caps = DesiredCapabilities::chrome(); $driver = RemoteWebDriver::create("http://localhost:4444/wd/hub", $caps, 120000); ``` **After:** ```php $caps = DesiredCapabilities::chrome(); $caps->setCapability('platformName', 'WIN11'); $caps->setCapability('browserVersion', 'latest'); $caps->setCapability('tb:options', [ 'key' => 'API_KEY', 'secret' => 'API_SECRET' ]); $driver = RemoteWebDriver::create("https://hub.testingbot.com/wd/hub", $caps, 120000); ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ```php $driver = RemoteWebDriver::create("https://hub.testingbot.com/wd/hub", $caps, 120000); ``` To see how to do this, please select a combination of browser, version and platform in the drop-down menus below. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a PHP WebDriver test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```php setCapability('platformName', 'WIN11'); $caps->setCapability('browserVersion', 'latest'); $caps->setCapability('tb:options', [ 'name' => $this->name() ]); // Point to the tunnel running on localhost:4445 $this->driver = RemoteWebDriver::create( 'http://localhost:4445/wd/hub', $caps, 120000 ); } protected function tearDown(): void { $this->driver->quit(); parent::tearDown(); } public function testTitle(): void { $this->driver->get('https://www.google.com/'); $this->assertEquals('Google', $this->driver->getTitle()); } } ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. To run tests in parallel, we recommend using [Paratest](https://github.com/brianium/paratest), which makes it very easy to run multiple PHPUnit tests simultaneously. ```bash vendor/bin/paratest -p 8 -f --phpunit=vendor/bin/phpunit WebDriverTest.php ``` The command above will run your tests in 8 separate processes (8 tests at once). ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. You can use our [PHP API client](https://github.com/testingbot/testingbot-php) to report back test results. ```php $api = new TestingBot\TestingBotAPI("API_KEY", "API_SECRET"); $api->updateJob($driver->getSessionID(), ['name' => 'mytest', 'success' => true]); ``` ## Other PHP Framework examples - [Behat](https://testingbot.com/support/web-automate/selenium/php/behat-mink) Behat is a BDD framework which runs on PHP. Mink is used for its browser emulation and works nicely together with Behat. - [Codeception](https://testingbot.com/support/web-automate/selenium/php/codeception) Codeception is a BDD-styled PHP testing framework. This testing framework offers good Selenium support. - [Laravel Dusk](https://testingbot.com/support/web-automate/selenium/php/laravel-dusk) Laravel Dusk provides an easy-to-use browser automation testing framework. - [PHPUnit](https://testingbot.com/support/web-automate/selenium/php/phpunit) PHPUnit is the most popular unit testing framework for PHP. It comes with good Selenium WebDriver support and is easy to set up. - [SimpleTest](https://testingbot.com/support/web-automate/selenium/php/simpletest) SimpleTest is a framework for unit testing, web site testing and mock objects for PHP. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/php/simpletest # SimpleTest Automated Testing SimpleTest's last release was in 2012 and the project is no longer maintained. For new PHP projects we recommend [PHPUnit](https://testingbot.com/support/web-automate/selenium/php/phpunit), [Codeception](https://testingbot.com/support/web-automate/selenium/php/codeception), or [Laravel Dusk](https://testingbot.com/support/web-automate/selenium/php/laravel-dusk) instead. You can find more information about SimpleTest on the [SimpleTest Repo](https://github.com/simpletest/simpletest). To use the example below, you'll need to download [php-webdriver](https://github.com/php-webdriver/php-webdriver) as well. ```php setCapability('platformName', 'WIN11'); $caps->setCapability('browserVersion', 'latest'); $caps->setCapability('tb:options', [ 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'SimpleTest Example' ]); $driver = RemoteWebDriver::create( 'https://hub.testingbot.com/wd/hub', $caps, 120000 ); $driver->get('https://www.google.com'); // SimpleTest's assertion method is assertEqual (singular), not assertEquals. $this->assertEqual('Google', $driver->getTitle()); $element = $driver->findElement(WebDriverBy::name('q')); if ($element) { $element->sendKeys('TestingBot'); $element->submit(); } $driver->quit(); } } ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ```php $caps = DesiredCapabilities::chrome(); $caps->setCapability('platformName', 'WIN11'); $caps->setCapability('browserVersion', 'latest'); $caps->setCapability('tb:options', [ 'key' => 'API_KEY', 'secret' => 'API_SECRET' ]); $driver = RemoteWebDriver::create( 'https://hub.testingbot.com/wd/hub', $caps, 120000 ); ``` To see how to do this, please select a combination of browser, version and platform in the drop-down menus below. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a SimpleTest WebDriver test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```php setCapability('platformName', 'WIN11'); $caps->setCapability('browserVersion', 'latest'); $caps->setCapability('tb:options', [ 'name' => 'SimpleTest Tunnel Example' ]); // Point to the tunnel running on localhost:4445 $driver = RemoteWebDriver::create( 'http://localhost:4445/wd/hub', $caps, 120000 ); $driver->get('https://www.google.com'); $this->assertEqual('Google', $driver->getTitle()); $element = $driver->findElement(WebDriverBy::name('q')); if ($element) { $element->sendKeys('TestingBot'); $element->submit(); } $driver->quit(); } } ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. SimpleTest does not ship with a parallel runner. Invoke the test script multiple times in parallel from the shell (for example with `xargs -P` or `GNU parallel`), passing different browser capabilities via environment variables: ```bash printf "chrome\nfirefox\nMicrosoftEdge\n" | xargs -P 3 -I {} sh -c 'BROWSER={} php TestOfLogging.php' ``` If you want a parallel runner with built-in worker management, consider switching to [PHPUnit + Paratest](https://testingbot.com/support/web-automate/selenium/php/phpunit). ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. You can use our [PHP API client](https://github.com/testingbot/testingbot-php) to report back test results. ```php $api = new TestingBot\TestingBotAPI('API_KEY', 'API_SECRET'); $api->updateJob($driver->getSessionID(), [ 'name' => 'My Test', 'success' => true, 'build' => 'build-123' ]); ``` ## Other PHP Framework examples - [Behat](https://testingbot.com/support/web-automate/selenium/php/behat-mink) Behat is a BDD framework which runs on PHP. Mink is used for its browser emulation and works nicely together with Behat. - [Codeception](https://testingbot.com/support/web-automate/selenium/php/codeception) Codeception is a BDD-styled PHP testing framework. This testing framework offers good Selenium support. - [Laravel Dusk](https://testingbot.com/support/web-automate/selenium/php/laravel-dusk) Laravel Dusk provides an easy-to-use browser automation testing framework. - [PHPUnit](https://testingbot.com/support/web-automate/selenium/php/phpunit) PHPUnit is the most popular unit testing framework for PHP. It comes with good Selenium WebDriver support and is easy to set up. - [SimpleTest](https://testingbot.com/support/web-automate/selenium/php/simpletest) SimpleTest is a framework for unit testing, web site testing and mock objects for PHP. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/java ### Java Examples: - [JUnit](https://testingbot.com/support/web-automate/selenium/java/junit) - [Parallel JUnit](https://testingbot.com/support/web-automate/selenium/java/parallel-junit) - [Selenide](https://testingbot.com/support/web-automate/selenium/java/selenide) - [TestNG](https://testingbot.com/support/web-automate/selenium/java/testng) - [TestNG + Cucumber](https://testingbot.com/support/web-automate/selenium/java/testng-cucumber) # Java Automated Testing Let's start with making sure Java is available on your system. We recommend Java 11 or newer (LTS versions like 11, 17 or 21). **For Windows:** - Download Java from [Eclipse Adoptium](https://adoptium.net/) or [Oracle](https://www.oracle.com/java/technologies/downloads/). - Run the installer and follow the setup wizard. - Verify the installation: `java -version` **For Linux (Debian/Ubuntu):** ```bash sudo apt-get install openjdk-17-jdk ``` **For macOS:** ```bash brew install openjdk@17 ``` Alternatively, download from [Eclipse Adoptium](https://adoptium.net/). ## Example With TestingBot you can easily run your automated tests with [any Java test framework](https://testingbot.com#frameworks). Here's a simple example to get started: ### Dependencies Add the Selenium dependency to your project: **Maven (pom.xml):** ```xml org.seleniumhq.selenium selenium-java 4.33.0 ``` **Gradle (build.gradle):** ```gradle implementation 'org.seleniumhq.selenium:selenium-java:4.33.0' ``` Alternatively, you can [download the JAR files](https://www.selenium.dev/downloads/) `(selenium-java-*.zip)` manually. ### Basic Test ```java import org.openqa.selenium.WebDriver; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URI; import java.util.HashMap; import java.util.Map; public class JavaSample { public static void main(String[] args) throws Exception { String key = System.getenv("TESTINGBOT_KEY"); String secret = System.getenv("TESTINGBOT_SECRET"); String hubURL = "https://" + key + ":" + secret + "@hub.testingbot.com/wd/hub"; ChromeOptions options = new ChromeOptions(); options.setPlatformName("WIN11"); options.setBrowserVersion("latest"); Map tbOptions = new HashMap<>(); tbOptions.put("name", "My First Test"); tbOptions.put("build", "build-1"); options.setCapability("tb:options", tbOptions); WebDriver driver = null; try { driver = new RemoteWebDriver(URI.create(hubURL).toURL(), options); driver.get("https://testingbot.com"); System.out.println("Page title is: " + driver.getTitle()); } finally { if (driver != null) { driver.quit(); } } } } ``` Set the `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` environment variables before running the test. You can find these on your [account page](https://testingbot.com/members/user/edit). This test will start a remote Chrome browser on Windows 11, open the TestingBot website and print the title of the page. This test does not make any assertions or verifications, but it's a quick intro in how to run a Java test on TestingBot. Make sure to always stop your test (`driver.quit()`), otherwise it will continue running, leading to a timeout. ### Configuring capabilities To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before:** ```java WebDriver driver = new FirefoxDriver(); ``` **After:** ```java FirefoxOptions options = new FirefoxOptions(); WebDriver driver = new RemoteWebDriver( new URL("https://key:secret@hub.testingbot.com/wd/hub"), options ); ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a Java test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```java import org.openqa.selenium.WebDriver; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URI; import java.util.HashMap; import java.util.Map; public class JavaSample { public static void main(String[] args) throws Exception { ChromeOptions options = new ChromeOptions(); options.setPlatformName("WIN11"); options.setBrowserVersion("latest"); Map tbOptions = new HashMap<>(); tbOptions.put("name", "Tunnel Test"); tbOptions.put("build", "build-1"); options.setCapability("tb:options", tbOptions); WebDriver driver = null; try { driver = new RemoteWebDriver(URI.create("http://localhost:4445/wd/hub").toURL(), options); driver.get("http://localhost:3000"); // Your internal website System.out.println("Page title is: " + driver.getTitle()); } finally { if (driver != null) { driver.quit(); } } } } ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. Please see our [Parallel JUnit documentation](https://testingbot.com/support/web-automate/selenium/java/parallel-junit) for parallel testing. ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark Tests as Passed or Failed To mark your test as passed or failed, or to send additional metadata to TestingBot, you can use our [API](https://testingbot.com/support/api). We provide a [Java client library](https://github.com/testingbot/testingbot-java) to help you interact with the TestingBot API easily. **Maven:** ```xml com.testingbot testingbotrest 1.0.10 ``` **Gradle:** ```gradle implementation 'com.testingbot:testingbotrest:1.0.10' ``` Once integrated into your test suite, you can programmatically update the test status and send metadata such as test name, build ID, or tags to TestingBot. ```java import com.testingbot.comingbotrest.TestingbotREST; import org.junit.After; import org.openqa.selenium.remote.RemoteWebDriver; import java.util.HashMap; import java.util.Map; @After public void tearDown() throws Exception { TestingbotREST api = new TestingbotREST(System.getenv("TESTINGBOT_KEY"), System.getenv("TESTINGBOT_SECRET")); Map data = new HashMap<>(); data.put("success", "1"); data.put("name", "My Test"); api.updateTest(((RemoteWebDriver) driver).getSessionId().toString(), data); driver.quit(); } ``` ## Other Java Framework examples - [JUnit](https://testingbot.com/support/web-automate/selenium/java/junit) JUnit is a unit testing framework for the Java programming language. - [Parallel JUnit](https://testingbot.com/support/web-automate/selenium/java/parallel-junit) By running multiple JUnit tests at the same time you can cut down on overall test time. - [TestNG](https://testingbot.com/support/web-automate/selenium/java/testng) TestNG is a framework similar to JUnit and NUnit, which supports some additional commands and features. - [TestNG + Cucumber](https://testingbot.com/support/web-automate/selenium/java/testng-cucumber) Run tests with TestNG and BDD Cucumber. - [Selenide](https://testingbot.com/support/web-automate/selenium/java/selenide) Selenide is a Java-based test framework that provides a simple and concise API for writing Selenium tests. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/java/junit # Set up your first Java test with JUnit See our [JUnit example repository](https://github.com/testingbot/java-junit-example) for a simple example on how to run JUnit tests in parallel on TestingBot. ## Dependencies Add the Selenium and JUnit dependencies to your project: **Maven (pom.xml):** ```xml org.seleniumhq.selenium selenium-java 4.33.0 junit junit 4.13.2 test ``` **Gradle (build.gradle):** ```gradle implementation 'org.seleniumhq.selenium:selenium-java:4.33.0' testImplementation 'junit:junit:4.13.2' ``` Below is a JUnit test example, showing you how to run a JUnit test on TestingBot ```java import org.junit.After; import org.junit.Before; import org.junit.Test; import static org.junit.Assert.assertEquals; import org.openqa.selenium.WebDriver; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URI; import java.time.Duration; import java.util.HashMap; import java.util.Map; public class SimpleTest { private WebDriver driver; @Before public void setUp() throws Exception { String key = System.getenv("TESTINGBOT_KEY"); String secret = System.getenv("TESTINGBOT_SECRET"); String hubURL = "https://" + key + ":" + secret + "@hub.testingbot.com/wd/hub"; ChromeOptions options = new ChromeOptions(); options.setPlatformName("WIN11"); options.setBrowserVersion("latest"); Map tbOptions = new HashMap<>(); tbOptions.put("name", "My JUnit Test"); tbOptions.put("build", "build-1"); options.setCapability("tb:options", tbOptions); driver = new RemoteWebDriver(URI.create(hubURL).toURL(), options); driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(30)); } @Test public void testSimple() throws Exception { driver.get("https://www.google.com"); assertEquals("Google", driver.getTitle()); } @After public void tearDown() throws Exception { if (driver != null) { driver.quit(); } } } ``` ## Configuring capabilities To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before:** ```java WebDriver driver = new FirefoxDriver(); ``` **After:** ```java FirefoxOptions options = new FirefoxOptions(); WebDriver driver = new RemoteWebDriver( new URL("https://key:secret@hub.testingbot.com/wd/hub"), options ); ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a JUnit test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```java import org.junit.After; import org.junit.Before; import org.junit.Test; import static org.junit.Assert.assertEquals; import org.openqa.selenium.WebDriver; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URI; import java.time.Duration; import java.util.HashMap; import java.util.Map; public class SimpleTest { private WebDriver driver; @Before public void setUp() throws Exception { ChromeOptions options = new ChromeOptions(); options.setPlatformName("WIN11"); options.setBrowserVersion("latest"); Map tbOptions = new HashMap<>(); tbOptions.put("name", "Tunnel Test"); tbOptions.put("build", "build-1"); options.setCapability("tb:options", tbOptions); driver = new RemoteWebDriver(URI.create("http://localhost:4445/wd/hub").toURL(), options); driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(30)); } @Test public void testSimple() throws Exception { driver.get("http://localhost:3000"); // Your internal website assertEquals("My App", driver.getTitle()); } @After public void tearDown() throws Exception { if (driver != null) { driver.quit(); } } } ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. Please see our [Parallel JUnit documentation](https://testingbot.com/support/web-automate/selenium/java/parallel-junit) for parallel testing. ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed To mark your test as passed or failed, or to send additional metadata to TestingBot, you can use our [API](https://testingbot.com/support/api). We provide a [Java client library](https://github.com/testingbot/testingbot-java) to help you interact with the TestingBot API easily. **Maven:** ```xml com.testingbot testingbotrest 1.0.10 ``` **Gradle:** ```gradle implementation 'com.testingbot:testingbotrest:1.0.10' ``` Once integrated into your test suite, you can programmatically update the test status and send metadata such as test name, build ID, or tags to TestingBot. ```java import com.testingbot.comingbotrest.TestingbotREST; import org.junit.After; import org.openqa.selenium.remote.RemoteWebDriver; import java.util.HashMap; import java.util.Map; @After public void tearDown() throws Exception { TestingbotREST api = new TestingbotREST(System.getenv("TESTINGBOT_KEY"), System.getenv("TESTINGBOT_SECRET")); Map data = new HashMap<>(); data.put("success", "1"); data.put("name", "My Test"); api.updateTest(((RemoteWebDriver) driver).getSessionId().toString(), data); driver.quit(); } ``` ## Other Java Framework examples - [JUnit](https://testingbot.com/support/web-automate/selenium/java/junit) JUnit is a unit testing framework for the Java programming language. - [Parallel JUnit](https://testingbot.com/support/web-automate/selenium/java/parallel-junit) By running multiple JUnit tests at the same time you can cut down on overall test time. - [TestNG](https://testingbot.com/support/web-automate/selenium/java/testng) TestNG is a framework similar to JUnit and NUnit, which supports some additional commands and features. - [TestNG + Cucumber](https://testingbot.com/support/web-automate/selenium/java/testng-cucumber) Run tests with TestNG and BDD Cucumber. - [Selenide](https://testingbot.com/support/web-automate/selenium/java/selenide) Selenide is a Java-based test framework that provides a simple and concise API for writing Selenium tests. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/java/parallel-junit # Speed up JUnit tests by running them in parallel By running multiple JUnit tests at the same time you can cut down on overall test time. ## Dependencies Add the following dependencies to your project. The example below uses `commons-io` for saving screenshots to disk: **Maven (pom.xml):** ```xml org.seleniumhq.selenium selenium-java 4.33.0 junit junit 4.13.2 test commons-io commons-io 2.16.1 ``` **Gradle (build.gradle):** ```gradle implementation 'org.seleniumhq.selenium:selenium-java:4.33.0' testImplementation 'junit:junit:4.13.2' implementation 'commons-io:commons-io:2.16.1' ``` ## Helper class needed to run JUnit tests in parallel ```java import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; import java.util.concurrent.TimeUnit; import org.junit.runners.Parameterized; import org.junit.runners.model.RunnerScheduler; public class Parallelized extends Parameterized { private static class ThreadPoolScheduler implements RunnerScheduler { private ExecutorService executor; public ThreadPoolScheduler() { String threads = System.getProperty("junit.parallel.threads", "16"); int numThreads = Integer.parseInt(threads); executor = Executors.newFixedThreadPool(numThreads); } @Override public void finished() { executor.shutdown(); try { executor.awaitTermination(10, TimeUnit.MINUTES); } catch (InterruptedException exc) { throw new RuntimeException(exc); } } @Override public void schedule(Runnable childStatement) { executor.submit(childStatement); } } public Parallelized(Class klass) throws Throwable { super(klass); setScheduler(new ThreadPoolScheduler()); } } ``` Below is a JUnit Test example, which uses the above helper class to run the test in parallel. ```java import java.io.File; import java.io.IOException; import java.net.URI; import java.util.HashMap; import java.util.LinkedList; import java.util.Map; import org.apache.commons.io.FileUtils; import org.junit.After; import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; import org.junit.runners.Parameterized; import org.openqa.selenium.MutableCapabilities; import org.openqa.selenium.OutputType; import org.openqa.selenium.TakesScreenshot; import org.openqa.selenium.WebDriver; import org.openqa.selenium.remote.RemoteWebDriver; @RunWith(Parallelized.class) public class JUnitParallel { private String platform; private String browserName; private String browserVersion; @Parameterized.Parameters public static LinkedList getEnvironments() throws Exception { LinkedList env = new LinkedList<>(); env.add(new String[]{"WIN11", "chrome", "latest"}); env.add(new String[]{"WIN11", "firefox", "latest"}); env.add(new String[]{"WIN11", "MicrosoftEdge", "latest"}); // Add more browser configurations here return env; } public JUnitParallel(String platform, String browserName, String browserVersion) { this.platform = platform; this.browserName = browserName; this.browserVersion = browserVersion; } private WebDriver driver; @Before public void setUp() throws Exception { String key = System.getenv("TESTINGBOT_KEY"); String secret = System.getenv("TESTINGBOT_SECRET"); String hubURL = "https://" + key + ":" + secret + "@hub.testingbot.com/wd/hub"; MutableCapabilities options = new MutableCapabilities(); options.setCapability("browserName", browserName); options.setCapability("browserVersion", browserVersion); options.setCapability("platformName", platform); Map tbOptions = new HashMap<>(); tbOptions.put("name", "Parallel JUnit Test"); tbOptions.put("build", "build-1"); options.setCapability("tb:options", tbOptions); driver = new RemoteWebDriver(URI.create(hubURL).toURL(), options); } @Test public void testSimple() throws Exception { driver.get("https://www.google.com"); String title = driver.getTitle(); System.out.println("Page title is: " + title); File srcFile = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE); try { FileUtils.copyFile(srcFile, new File("Screenshot.png")); } catch (IOException e) { e.printStackTrace(); } } @After public void tearDown() throws Exception { driver.quit(); } } ``` Run the tests with `mvn test`. Note that the Maven Surefire plugin only picks up test classes whose names match `*Test`, `Test*`, `*Tests` or `*TestCase` by default. The example class above is named `JUnitParallel`, so either rename it (for example `JUnitParallelTest`) or add an explicit `` entry to the Surefire configuration, otherwise `mvn test` will run zero tests. The number of parallel threads is controlled by the `junit.parallel.threads` system property (default `16` in the helper class above). Set this to match the number of parallel sessions allowed on your TestingBot plan to avoid queuing, for example `mvn test -Djunit.parallel.threads=5`. ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a JUnit test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```java @Before public void setUp() throws Exception { MutableCapabilities options = new MutableCapabilities(); options.setCapability("browserName", browserName); options.setCapability("browserVersion", browserVersion); options.setCapability("platformName", platform); Map tbOptions = new HashMap<>(); tbOptions.put("name", "Parallel Tunnel Test"); tbOptions.put("build", "build-1"); options.setCapability("tb:options", tbOptions); driver = new RemoteWebDriver(URI.create("http://localhost:4445/wd/hub").toURL(), options); } ``` ## Mark tests as passed/failed To see if a test passed or not in our member area, or to send additional meta-data to TestingBot, you need to use our [API](https://testingbot.com/support/api). TestingBot has a [Java client](https://github.com/testingbot/testingbot-java) for using the TestingBot API. **Maven:** ```xml com.testingbot testingbotrest 1.0.10 ``` **Gradle:** ```gradle implementation 'com.testingbot:testingbotrest:1.0.10' ``` Once included with your tests, you can send back test status and other meta-data to TestingBot: ```java import com.testingbot.comingbotrest.TestingbotREST; import org.junit.After; import org.openqa.selenium.remote.RemoteWebDriver; import java.util.HashMap; import java.util.Map; @After public void tearDown() throws Exception { TestingbotREST api = new TestingbotREST(System.getenv("TESTINGBOT_KEY"), System.getenv("TESTINGBOT_SECRET")); Map data = new HashMap<>(); data.put("success", "1"); data.put("name", "My Test"); api.updateTest(((RemoteWebDriver) driver).getSessionId().toString(), data); driver.quit(); } ``` ## Other Java Framework examples - [JUnit](https://testingbot.com/support/web-automate/selenium/java/junit) JUnit is a unit testing framework for the Java programming language. - [Parallel JUnit](https://testingbot.com/support/web-automate/selenium/java/parallel-junit) By running multiple JUnit tests at the same time you can cut down on overall test time. - [TestNG](https://testingbot.com/support/web-automate/selenium/java/testng) TestNG is a framework similar to JUnit and NUnit, which supports some additional commands and features. - [TestNG + Cucumber](https://testingbot.com/support/web-automate/selenium/java/testng-cucumber) Run tests with TestNG and BDD Cucumber. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/java/selenide # Selenide Automated Testing TestingBot has a [Selenide example GitHub project](https://github.com/testingbot/java-testng-selenide-example) available. This contains an example on how to run Selenide tests with TestingBot. Selenide is a Java-based test framework, capable of using Selenium WebDriver to test websites on (remote) browsers. You can use Selenide in combination with a test framework such as JUnit, TestNG, Cucumber and others. The syntax used by Selenide is short, it uses a concise API that makes your test code shorter and perhaps more readable. To get started, let's go over all the necessary steps required to run your first test with TestNG and Selenide, using TestingBot's remote browsers. ## Prerequisites Let's start with making sure Java is available on your system. Starting with Selenide 7, at least Java 17 is required. **For Windows:** - You can download Java for Windows from [Java.com](https://www.java.com/en/download/manual.jsp) - Run the installer and follow the setup wizard to install Java. **For Linux (Debian/Ubuntu):** ```bash sudo apt-get install openjdk-17-jdk ``` **For macOS:** ```bash brew install openjdk@17 ``` Alternatively, download from [Eclipse Adoptium](https://adoptium.net/). ## Set up pom.xml dependencies Now we can specify which dependencies we need to be able to run a test. Create a new file called `pom.xml` and make sure to define at least the following dependencies: - `com.codeborne:selenide` - `org.testng:testng` - `org.seleniumhq.selenium:selenium-java` ```xml 4.0.0 testingbot-testng-selenide com.testingbot 1.0-SNAPSHOT jar testingbot_selenide_testng An example project to run Selenide + TestNG tests on TestingBot's browser grid 17 17 com.codeborne selenide 7.9.3 test org.testng testng 7.11.0 test org.seleniumhq.selenium selenium-java 4.33.0 test maven-compiler-plugin 3.13.0 ${maven.compiler.source} ${maven.compiler.target} org.apache.maven.plugins maven-surefire-plugin 3.2.5 classes 40 false ``` ## Define TestingBot driver We need to define a base class to let Selenide know how to connect to the TestingBot browser grid. ```java package com.testingbotdemo.selenium.selenide.drivers; import java.net.URI; import java.net.MalformedURLException; import java.util.HashMap; import java.util.Map; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import org.testng.annotations.AfterMethod; import org.testng.annotations.BeforeMethod; import com.codeborne.selenide.WebDriverRunner; public class TestingBotDriver { protected RemoteWebDriver driver; @BeforeMethod(alwaysRun = true) public void setUp() throws Exception { String key = System.getenv("TESTINGBOT_KEY"); String secret = System.getenv("TESTINGBOT_SECRET"); String hubURL = "https://" + key + ":" + secret + "@hub.testingbot.com/wd/hub"; ChromeOptions options = new ChromeOptions(); options.setPlatformName("WIN11"); options.setBrowserVersion("latest"); Map tbOptions = new HashMap<>(); tbOptions.put("name", "Selenide Example"); tbOptions.put("build", "build-1"); options.setCapability("tb:options", tbOptions); try { driver = new RemoteWebDriver(URI.create(hubURL).toURL(), options); WebDriverRunner.setWebDriver(driver); } catch (MalformedURLException e) { e.printStackTrace(); throw new RuntimeException("Invalid URL provided for TestingBot hub", e); } } @AfterMethod(alwaysRun = true) public void tearDown() throws Exception { if (driver != null) { driver.quit(); } } } ``` The `ChromeOptions` class is used to define which remote browser you want to use on TestingBot. ## Test Now we can create our first Selenide test. Create a new class that extends from `TestingBotDriver` with the following contents: ```java package com.testingbotdemo.selenium.selenide.drivers; import static com.codeborne.selenide.Selenide.*; import static com.codeborne.selenide.Condition.*; import org.testng.annotations.Test; public class ExampleTest extends TestingBotDriver { @Test public void test() throws Exception { open("https://testingbot.com"); $("body").shouldHave(text("TestingBot")); } } ``` ## Run Selenide test You can now run the Selenide test from the commandline: ```bash TESTINGBOT_KEY={..your-key..} TESTINGBOT_SECRET={..your-secret..} mvn test ``` The Selenide test will run on a remote TestingBot browser. Once the test has finished, you will be able to see the result, together with a video and logs in the TestingBot dashboard. ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a TestNG test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back. Replace the contents of your existing `TestingBotDriver` class with the version below (it uses the same package and class name): ```java package com.testingbotdemo.selenium.selenide.drivers; import java.net.URI; import java.net.MalformedURLException; import java.util.HashMap; import java.util.Map; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import org.testng.annotations.AfterMethod; import org.testng.annotations.BeforeMethod; import com.codeborne.selenide.WebDriverRunner; public class TestingBotDriver { protected RemoteWebDriver driver; @BeforeMethod(alwaysRun = true) public void setUp() throws Exception { ChromeOptions options = new ChromeOptions(); options.setPlatformName("WIN11"); options.setBrowserVersion("latest"); Map tbOptions = new HashMap<>(); tbOptions.put("name", "Tunnel Selenide Example"); tbOptions.put("build", "build-1"); options.setCapability("tb:options", tbOptions); driver = new RemoteWebDriver(URI.create("http://localhost:4445/wd/hub").toURL(), options); WebDriverRunner.setWebDriver(driver); } @AfterMethod(alwaysRun = true) public void tearDown() throws Exception { if (driver != null) { driver.quit(); } } } ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. To run a test on different browsers at the same time, you can run the same `mvn test` command multiple times in parallel, each with a different browser/capability environment variable. To do this, replace the hard-coded `ChromeOptions` block in `TestingBotDriver.setUp()` with capabilities read from environment variables (remember to import `org.openqa.selenium.MutableCapabilities`): ```java MutableCapabilities options = new MutableCapabilities(); options.setCapability("browserName", System.getenv("CAPABILITY_BROWSERNAME")); options.setCapability("platformName", System.getenv("CAPABILITY_PLATFORMNAME")); options.setCapability("browserVersion", System.getenv("CAPABILITY_BROWSERVERSION")); ``` Now you can run the same test on multiple browsers simultaneously: ```bash CAPABILITY_BROWSERNAME=chrome CAPABILITY_BROWSERVERSION=latest CAPABILITY_PLATFORMNAME=WIN11 mvn test & CAPABILITY_BROWSERNAME=safari CAPABILITY_BROWSERVERSION=latest CAPABILITY_PLATFORMNAME=SONOMA mvn test & ``` ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed To see if a test passed or not in our member area, or to send additional meta-data to TestingBot, you can use the TestingBot [API](https://testingbot.com/support/api). TestingBot has a [Java client](https://github.com/testingbot/testingbot-java) to facilitate interacting with the TestingBot API. **Maven:** ```xml com.testingbot testingbotrest 1.0.10 ``` **Gradle:** ```gradle implementation 'com.testingbot:testingbotrest:1.0.10' ``` Once you've included this client with your tests, you will be able to send back the test status and other meta-data to TestingBot: ```java import com.testingbot.comingbotrest.TestingbotREST; import org.testng.annotations.AfterMethod; import java.util.HashMap; import java.util.Map; @AfterMethod public void tearDown() throws Exception { TestingbotREST api = new TestingbotREST(System.getenv("TESTINGBOT_KEY"), System.getenv("TESTINGBOT_SECRET")); Map data = new HashMap<>(); data.put("success", "1"); data.put("name", "My Test"); api.updateTest(driver.getSessionId().toString(), data); driver.quit(); } ``` ## Other Java Framework examples - [JUnit](https://testingbot.com/support/web-automate/selenium/java/junit) JUnit is a unit testing framework for the Java programming language. - [Parallel JUnit](https://testingbot.com/support/web-automate/selenium/java/parallel-junit) By running multiple JUnit tests at the same time you can cut down on overall test time. - [TestNG](https://testingbot.com/support/web-automate/selenium/java/testng) TestNG is a framework similar to JUnit and NUnit, which supports some additional commands and features. - [TestNG + Cucumber](https://testingbot.com/support/web-automate/selenium/java/testng-cucumber) Run tests with TestNG and BDD Cucumber. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/java/testng # TestNG Automated Testing See our [TestNG example repository](https://github.com/testingbot/java-testng-example) for a simple example on how to run TestNG tests on TestingBot. TestNG is a framework similar to JUnit and NUnit, which supports some additional commands and features. You can find more info about TestNG on the [TestNG website](https://testng.org/#_testng_documentation). To run your first test with TestNG, please follow the simple example below. ## Dependencies Add the Selenium and TestNG dependencies to your project. The parallel example further down also uses `commons-io` to save screenshots: **Maven (pom.xml):** ```xml org.seleniumhq.selenium selenium-java 4.33.0 org.testng testng 7.11.0 test commons-io commons-io 2.16.1 ``` **Gradle (build.gradle):** ```gradle implementation 'org.seleniumhq.selenium:selenium-java:4.33.0' testImplementation 'org.testng:testng:7.11.0' implementation 'commons-io:commons-io:2.16.1' ``` ## Example code ```java import java.net.URI; import java.util.HashMap; import java.util.Map; import org.openqa.selenium.By; import org.openqa.selenium.WebDriver; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import org.testng.Assert; import org.testng.annotations.AfterClass; import org.testng.annotations.BeforeClass; import org.testng.annotations.Test; public class TestingBotTest { private WebDriver driver; @BeforeClass public void setUp() throws Exception { String key = System.getenv("TESTINGBOT_KEY"); String secret = System.getenv("TESTINGBOT_SECRET"); String hubURL = "https://" + key + ":" + secret + "@hub.testingbot.com/wd/hub"; ChromeOptions options = new ChromeOptions(); options.setPlatformName("WIN11"); options.setBrowserVersion("latest"); Map tbOptions = new HashMap<>(); tbOptions.put("name", "My TestNG Test"); tbOptions.put("build", "build-1"); options.setCapability("tb:options", tbOptions); driver = new RemoteWebDriver(URI.create(hubURL).toURL(), options); } @Test public void testSimple() throws Exception { driver.get("https://testingbot.com/"); String searchHeader = driver.findElement(By.cssSelector("H1")) .getText().toLowerCase(); Assert.assertTrue(searchHeader.contains("cross browser testing")); } @AfterClass public void tearDown() throws Exception { driver.quit(); } } ``` ## Configuring capabilities To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before:** ```java WebDriver driver = new FirefoxDriver(); ``` **After:** ```java FirefoxOptions options = new FirefoxOptions(); WebDriver driver = new RemoteWebDriver( new URL("https://API_KEY:API_SECRET@hub.testingbot.com/wd/hub"), options ); ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a TestNG test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```java import java.net.URI; import java.util.HashMap; import java.util.Map; import org.openqa.selenium.By; import org.openqa.selenium.WebDriver; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import org.testng.Assert; import org.testng.annotations.AfterClass; import org.testng.annotations.BeforeClass; import org.testng.annotations.Test; public class TestingBotTest { private WebDriver driver; @BeforeClass public void setUp() throws Exception { ChromeOptions options = new ChromeOptions(); options.setPlatformName("WIN11"); options.setBrowserVersion("latest"); Map tbOptions = new HashMap<>(); tbOptions.put("name", "Tunnel Test"); tbOptions.put("build", "build-1"); options.setCapability("tb:options", tbOptions); driver = new RemoteWebDriver(URI.create("http://localhost:4445/wd/hub").toURL(), options); } @Test public void testSimple() throws Exception { driver.get("http://localhost:3000"); // Your internal website String header = driver.findElement(By.cssSelector("H1")) .getText().toLowerCase(); Assert.assertTrue(header.contains("welcome")); } @AfterClass public void tearDown() throws Exception { driver.quit(); } } ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. To run a test on different browsers at the same time, you will need to create a `testng.xml` file and create a testcase that uses parameters (`org.testng.annotations.Parameters`). Please see the example below: ```java package tb; import org.testng.annotations.Test; import org.testng.annotations.AfterClass; import org.testng.annotations.BeforeClass; import org.testng.Assert; import java.io.File; import java.io.IOException; import java.net.URI; import java.util.HashMap; import java.util.Map; import org.apache.commons.io.FileUtils; import org.openqa.selenium.By; import org.openqa.selenium.MutableCapabilities; import org.openqa.selenium.OutputType; import org.openqa.selenium.TakesScreenshot; import org.openqa.selenium.WebDriver; import org.openqa.selenium.WebElement; import org.openqa.selenium.remote.RemoteWebDriver; public class TestCase { private WebDriver driver; @BeforeClass @org.testng.annotations.Parameters(value={"browser","version","platform"}) public void setUp(String browser, String version, String platform) throws Exception { String key = System.getenv("TESTINGBOT_KEY"); String secret = System.getenv("TESTINGBOT_SECRET"); String hubURL = "https://" + key + ":" + secret + "@hub.testingbot.com/wd/hub"; MutableCapabilities options = new MutableCapabilities(); options.setCapability("browserName", browser); options.setCapability("browserVersion", version); options.setCapability("platformName", platform); Map tbOptions = new HashMap<>(); tbOptions.put("name", "Parallel TestNG"); tbOptions.put("build", "build-1"); options.setCapability("tb:options", tbOptions); driver = new RemoteWebDriver(URI.create(hubURL).toURL(), options); } @Test public void testSimple() throws Exception { driver.get("https://www.google.com"); System.out.println("Page title is: " + driver.getTitle()); Assert.assertEquals("Google", driver.getTitle()); WebElement element = driver.findElement(By.name("q")); element.sendKeys("TestingBot"); element.submit(); File srcFile = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE); try { FileUtils.copyFile(srcFile, new File("Screenshot.png")); } catch (IOException e) { e.printStackTrace(); } } @AfterClass public void tearDown() throws Exception { driver.quit(); } } ``` Now you need to create a `testng.xml` file which will provide the test cases with parameters (browser combinations): ```xml ``` ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed To see if a test passed or not in our member area, or to send additional meta-data to TestingBot, you need to use our [API](https://testingbot.com/support/api). TestingBot has a [Java client](https://github.com/testingbot/testingbot-java) for using the TestingBot API. **Maven:** ```xml com.testingbot testingbotrest 1.0.10 ``` **Gradle:** ```gradle implementation 'com.testingbot:testingbotrest:1.0.10' ``` Once included with your tests, you can send back test status and other meta-data to TestingBot: ```java import com.testingbot.comingbotrest.TestingbotREST; import org.testng.annotations.AfterClass; import org.openqa.selenium.remote.RemoteWebDriver; import java.util.HashMap; import java.util.Map; @AfterClass public void tearDown() throws Exception { TestingbotREST api = new TestingbotREST(System.getenv("TESTINGBOT_KEY"), System.getenv("TESTINGBOT_SECRET")); Map data = new HashMap<>(); data.put("success", "1"); data.put("name", "My Test"); api.updateTest(((RemoteWebDriver) driver).getSessionId().toString(), data); driver.quit(); } ``` ## More TestNG resources Looking for more information regarding TestNG? See our [TestNG with Selenium](https://testingbot.com/resources/articles/testng-selenium) article for more information on what TestNG supports, including assertions and annotations. ## Other Java Framework examples - [JUnit](https://testingbot.com/support/web-automate/selenium/java/junit) JUnit is a unit testing framework for the Java programming language. - [Parallel JUnit](https://testingbot.com/support/web-automate/selenium/java/parallel-junit) By running multiple JUnit tests at the same time you can cut down on overall test time. - [TestNG + Cucumber](https://testingbot.com/support/web-automate/selenium/java/testng-cucumber) Run tests with TestNG and BDD Cucumber. - [Selenide](https://testingbot.com/support/web-automate/selenium/java/selenide) Selenide is a Java-based test framework, with a concise API for writing stable and maintainable UI tests. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/java/testng-cucumber # TestNG & Cucumber Automated Testing Please consult the [TestNG + Cucumber example repository](https://github.com/testingbot/java-testng-cucumber-example) to see a simple example on how to run TestNG & Cucumber tests on TestingBot. TestNG is a powerful testing framework inspired by JUnit and NUnit but with more flexible and robust features. It is widely used by Java enthusiasts for testing because it offers powerful annotations, test configuration and parallel testing. Cucumber is a popular tool for Behavior-Driven Development (BDD), allowing test scripts to be written in a human-readable format using the Gherkin language. Because Cucumber uses this natural-like language, it enables collaboration between technical and non-technical team members. To get started, let's go through all the necessary steps required to run your first test with TestNG and Cucumber on TestingBot. ## Prerequisites Let's start with making sure Java is available on your system. **For Windows:** - You can download Java for Windows from [Java.com](https://www.java.com/en/download/manual.jsp) - Run the installer and follow the setup wizard to install Java. **For Linux (Debian/Ubuntu):** ```bash sudo apt-get install openjdk-17-jdk ``` **For macOS:** ```bash brew install openjdk@17 ``` Alternatively, download from [Eclipse Adoptium](https://adoptium.net/). ## Set up pom.xml dependencies Now we can specify which dependencies we need to be able to run a test. Create a new file called `pom.xml` and make sure to define at least the following dependencies: - `io.cucumber:cucumber-java` `io.cucumber:cucumber-testng` `org.testng:testng` `org.seleniumhq.selenium:selenium-java` ```xml 4.0.0 testingbot-testng-cucumber com.testingbot 1.0-SNAPSHOT jar testingbot_cucumber_testng An example project to run Cucumber + TestNG tests on TestingBot's browser grid io.cucumber cucumber-java 7.22.1 io.cucumber cucumber-testng 7.22.1 org.testng testng 7.11.0 test org.seleniumhq.selenium selenium-java 4.33.0 test maven-compiler-plugin 3.13.0 17 17 org.apache.maven.plugins maven-surefire-plugin 3.2.5 classes 40 false ``` ## Project Structure Now we need to create a folder structure that will contain the `.feature` files and step definitions. Create a folder for the `.feature` files: ```bash mkdir -p src/test/resources/features ``` We can also define directories for the step definitions, driver and the TestingBot runner class: ```bash mkdir -p src/test/java/com/testingbotdemo/selenium/cucumber/{definitions,drivers,runners} ``` ## Define TestingBot driver Now we can define a custom driver that will run the tests on the TestingBot grid: ```java package com.testingbotdemo.selenium.cucumber.drivers; import org.openqa.selenium.WebDriver; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URI; import java.net.MalformedURLException; import java.util.HashMap; import java.util.Map; public class TestingBotDriver { public static WebDriver getDriver() { String key = System.getenv("TESTINGBOT_KEY"); String secret = System.getenv("TESTINGBOT_SECRET"); String hubURL = "https://" + key + ":" + secret + "@hub.testingbot.com/wd/hub"; ChromeOptions options = new ChromeOptions(); options.setPlatformName("WIN11"); options.setBrowserVersion("latest"); Map tbOptions = new HashMap<>(); tbOptions.put("name", "Cucumber Example"); tbOptions.put("build", "build-1"); options.setCapability("tb:options", tbOptions); try { return new RemoteWebDriver(URI.create(hubURL).toURL(), options); } catch (MalformedURLException e) { e.printStackTrace(); throw new RuntimeException("Invalid URL provided for TestingBot hub", e); } } } ``` We will need to make sure to later specify the `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` environment variables to authenticate and run tests on the TestingBot remote browser grid. The `ChromeOptions` class is used to define which remote browser we would like to run this Cucumber test on. ## Test Runner Next, let's create a Test runner class that will use the `CucumberOptions` annotation to let Cucumber know where to find the feature files and the step definitions. ```java package com.testingbotdemo.selenium.cucumber.runners; import io.cucumber.testng.AbstractTestNGCucumberTests; import io.cucumber.testng.CucumberOptions; @CucumberOptions( features = "src/test/resources/features", glue = "com.testingbotdemo.selenium.cucumber.definitions" ) public class TestRunner extends AbstractTestNGCucumberTests { } ``` ## Step Definitions Finally, we need to define the step definitions. This is the logic that converts the Gherkin syntax to actual code. The code that we will use will interact with the [Driver class](https://testingbot.com#driver) we defined. ```java package com.testingbotdemo.selenium.cucumber.definitions; import io.cucumber.java.After; import io.cucumber.java.Before; import io.cucumber.java.en.Given; import io.cucumber.java.en.Then; import io.cucumber.java.en.When; import org.openqa.selenium.By; import org.openqa.selenium.Keys; import org.testng.Assert; import com.testingbotdemo.selenium.cucumber.drivers.TestingBotDriver; import org.openqa.selenium.WebDriver; public class StepDefinitions { private WebDriver driver; @Before public void setUp() { driver = TestingBotDriver.getDriver(); } @Given("I open Google") public void i_open_google() { driver.get("https://www.google.com"); } @When("I search for {string}") public void i_search_for(String query) { driver.findElement(By.name("q")).sendKeys(query + Keys.ENTER); } @Then("I should see results") public void i_should_see_results() { Assert.assertTrue(driver.getTitle().contains("Google")); } @After public void tearDown() { if (driver != null) { driver.quit(); } } } ``` ## Feature file Finally, create a `.feature` file that describes the scenario in Gherkin. The steps below map to the step definitions defined above. Save it as `src/test/resources/features/search.feature` (the same `features` directory referenced by the `@CucumberOptions` in the runner): ```gherkin Feature: Google Search Scenario: Search for TestingBot Given I open Google When I search for "TestingBot" Then I should see results ``` Without at least one feature file in this directory, Cucumber will not discover any scenarios and `mvn test` will pass without running your step definitions. ## Run Cucumber test You can now run the Cucumber test: ```bash TESTINGBOT_KEY={..your-key..} TESTINGBOT_SECRET={..your-secret..} mvn test ``` The test will run on a remote TestingBot browser. Once the test has finished, you will be able to see the result, together with a video and logs in the TestingBot dashboard. ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a TestNG test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back. Replace the contents of your existing `TestingBotDriver` class with the version below (it uses the same package and class name): ```java package com.testingbotdemo.selenium.cucumber.drivers; import org.openqa.selenium.WebDriver; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URI; import java.net.MalformedURLException; import java.util.HashMap; import java.util.Map; public class TestingBotDriver { public static WebDriver getDriver() { ChromeOptions options = new ChromeOptions(); options.setPlatformName("WIN11"); options.setBrowserVersion("latest"); Map tbOptions = new HashMap<>(); tbOptions.put("name", "Tunnel Cucumber Example"); tbOptions.put("build", "build-1"); options.setCapability("tb:options", tbOptions); try { return new RemoteWebDriver(URI.create("http://localhost:4445/wd/hub").toURL(), options); } catch (MalformedURLException e) { e.printStackTrace(); throw new RuntimeException("Invalid URL provided for TestingBot tunnel", e); } } } ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. To run a test on different browsers at the same time, you can run the same `mvn test` command multiple times in parallel, each with a different browser/capability passed as a system property. For example, assuming you read the desired capabilities from system properties (set via `-D` on the command line): ```java ChromeOptions options = new ChromeOptions(); options.setPlatformName(System.getProperty("CAPABILITY_PLATFORMNAME")); options.setBrowserVersion(System.getProperty("CAPABILITY_BROWSERVERSION")); ``` Now you can run the same test on multiple browsers simultaneously: ```bash mvn test -DCAPABILITY_BROWSERNAME=chrome -DCAPABILITY_BROWSERVERSION=latest -DCAPABILITY_PLATFORMNAME=WIN11 mvn test -DCAPABILITY_BROWSERNAME=safari -DCAPABILITY_BROWSERVERSION=latest -DCAPABILITY_PLATFORMNAME=SONOMA ``` ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed To see if a test passed or not in our member area, or to send additional meta-data to TestingBot, you can use the TestingBot [API](https://testingbot.com/support/api). TestingBot has a [Java client](https://github.com/testingbot/testingbot-java) to facilitate interacting with the TestingBot API. **Maven:** ```xml com.testingbot testingbotrest 1.0.10 ``` **Gradle:** ```gradle implementation 'com.testingbot:testingbotrest:1.0.10' ``` Once you've included this client with your tests, you will be able to send back the test status and other meta-data to TestingBot: ```java import com.testingbot.comingbotrest.TestingbotREST; import io.cucumber.java.After; import io.cucumber.java.Scenario; import java.util.HashMap; import java.util.Map; import org.openqa.selenium.remote.RemoteWebDriver; @After public void tearDown(Scenario scenario) throws Exception { TestingbotREST api = new TestingbotREST(System.getenv("TESTINGBOT_KEY"), System.getenv("TESTINGBOT_SECRET")); Map data = new HashMap<>(); data.put("success", scenario.isFailed() ? "0" : "1"); data.put("name", scenario.getName()); api.updateTest(((RemoteWebDriver) driver).getSessionId().toString(), data); driver.quit(); } ``` ## Other Java Framework examples - [JUnit](https://testingbot.com/support/web-automate/selenium/java/junit) JUnit is a unit testing framework for the Java programming language. - [Parallel JUnit](https://testingbot.com/support/web-automate/selenium/java/parallel-junit) By running multiple JUnit tests at the same time you can cut down on overall test time. - [TestNG](https://testingbot.com/support/web-automate/selenium/java/testng) TestNG is a framework similar to JUnit and NUnit, which supports some additional commands and features. - [Selenide](https://testingbot.com/support/web-automate/selenium/java/selenide) Selenide is a Java-based test framework, with a concise API for writing stable and maintainable UI tests. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/javascript # Javascript Unit Testing It's possible to run your javascript unit tests on [our cloud of browsers](https://testingbot.com/support/web-automate/browsers). Instead of running your javascript test on a couple of browsers, you can run your tests on dozens of different browsers/versions/platforms all in parallel on our grid. ## Javascript Test Frameworks Below is an overview of the javascript unit testing frameworks, each with an example. Other javascript testing frameworks not listed here also do work with TestingBot, we just don't have any example yet. ### Pick a Javascript test framework - [Karma](https://testingbot.com/support/web-automate/selenium/javascript/karma) Javascript testing with the Karma test runner. - [Intern](https://testingbot.com/support/web-automate/selenium/javascript/intern) Intern runs your javascript tests with full code coverage reporting. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/javascript/karma # Javascript unit testing with Karma We've prepared an example with full TestingBot integration which is easy to set up and run. With our integrated `karma-testingbot-launcher`, you can easily run tests with Karma on TestingBot. Our launcher will automatically start/stop a tunnel and report back the test results to us. ## Installing the example Make sure you have installed both `git` and `NodeJS`. Then proceed to clone our [karma-testingbot-example](https://github.com/testingbot/karma-testingbot-example) with the follow commands: ```bash git clone https://github.com/testingbot/karma-testingbot-example.git cd karma-testingbot-example ``` Once that's installed, please run the follow command to set up the example: ```bash sudo npm install -g karma-cli && npm install ``` ## Run Karma test with our [karma-testingbot-launcher](https://github.com/karma-runner/karma-testingbot-launcher) plugin In the example's directory, you'll find a file `karma.conf-testingbot.js` which contains a "customLaunchers" variable with multiple browsers defined. This is the place where you specify on which browsers you want your test to run. To actually run the test on TestingBot, we first need to specify the `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` environment variables, which you can find in our member area. ```bash export TESTINGBOT_KEY=YOUR_KEY export TESTINGBOT_SECRET=YOUR_SECRET ``` Now you're ready to run the test on TestingBot. To start the test, run the following command: ```bash karma start karma.conf-testingbot.js ``` This will download and start our [TestingBot Tunnel](https://testingbot.com/support/tunnel), run your test and report back the result. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/javascript/intern # NodeJS with Intern Intern has not seen significant releases since 2019. For new JavaScript end-to-end testing projects we recommend [WebdriverIO](https://testingbot.com/support/web-automate/selenium/nodejs/webdriverio) or [Nightwatch](https://testingbot.com/support/web-automate/selenium/nodejs/nightwatch). With Intern you can run Javascript unit tests on TestingBot's browser grid. Run your javascript test on all our [browsers](https://testingbot.com/support/web-automate/browsers) in the cloud. ## Installing the example Make sure you have installed `git` and `NodeJS`. Then proceed to clone our [intern\_example](https://github.com/testingbot/intern_example) with the follow commands: ```bash git clone https://github.com/testingbot/intern_example.git cd intern_example ``` Once that's installed, please run the follow command to set up the example: ```bash npm install intern --save-dev ``` ## Run Intern test on TestingBot In the `tests/intern.json` config you'll find `tunnel` and `environments` entries where you can indicate on which browsers you want your test to run on via TestingBot. To actually run the test on TestingBot, we first need to point the WebDriver tunnel at the TestingBot Selenium hub and pass the TestingBot `key` and `secret`. In `tests/intern.json`: ```javascript { "tunnel": "testingbot", "tunnelOptions": { "username": "YOUR_KEY", "accessKey": "YOUR_SECRET" }, "environments": [ { "browserName": "chrome", "browserVersion": "latest", "platformName": "WIN11" } ] } ``` Now you're ready to run the test on TestingBot. In Intern 4+, the runner is invoked as `intern` (the old `intern-runner` binary was removed). To start the test, run: ```bash ./node_modules/.bin/intern config=@testingbot ``` This will download and start our [TestingBot Tunnel](https://testingbot.com/support/tunnel), run your test and report back the result to TestingBot. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/ide # Selenium IDE Selenium IDE is a [Chrome addon](https://chrome.google.com/webstore/detail/selenium-ide/mooikfkahbdckldjjndioackbalphokd) / [Firefox addon](https://www.selenium.dev/selenium-ide/) which you can install to record and run browser tests. Once you've installed the plugin, you can record an entire flow from your own browser. You can record every interaction you do on a website and add assertions in your test. ## What can Selenium IDE do for me? Here's one example: you can record yourself clicking through various pages on your website. During this recording, you can right click various elements and indicate that these need to be present, or contain a specific value. ![Selenium IDE screenshot](https://testingbot.com/assets/wizard/ide-2e7510cfbc2b24f20c4483cb4045179fdf5036cdb391ae72ad470f8f360a3a18.jpg) Once you've finished this recording, you can choose to instantly playback the recording. The IDE will mimic your recorded behavior together with the assertions you've recorded. ## TestingBot integration with Selenium IDE When you finished recording a test, you can choose to save the test from the Selenium IDE. The saved file (with a `.side` extension) can be uploaded into [our TestLab](https://testingbot.com/support/web-automate/codeless-automation/add-test). In our TestLab, you can indicate on which browsers and platforms the test needs to run. You can schedule tests and indicate if you would like to be alerted when a test fails. ## Advanced: Selenium SIDE runner with TestingBot It's possible to run tests recorded with Selenium IDE via the [selenium-side-runner](https://github.com/SeleniumHQ/selenium-ide/tree/v3/packages/selenium-side-runner) commandline program on TestingBot. Simply save a recorded test as a `.side` file and run it like this: ```bash npm install -g selenium-side-runner selenium-side-runner --server https://key:secret@hub.testingbot.com/wd/hub -c "browserName=chrome platformName=WIN11 browserVersion=latest" *.side ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/katalon-studio # Katalon Studio See our [Katalon Studio Plugin](https://testingbot.com/support/integrations/katalon-studio) for easy integration between TestingBot and Katalon Studio. [Katalon Studio](https://www.katalon.com/katalon-studio/) allows testers to easily create, manage and run automated tests. Below is some more information on how to run your recorded tests via Katalon Studio on TestingBot and take advantage of the [large collection of browsers and mobile devices](https://testingbot.com/support/web-automate/browsers) available on the TestingBot cloud. ## Configuration For this tutorial, we assume you've already created or recorded a web/mobile test with Katalon Studio. To connect with TestingBot, you will need to configure Katalon Studio to pass the correct configuration to TestingBot. Go to **Project \> Settings** then choose the **Desired Capabilities** option and click **Remote**. Fill in the correct **Remote server URL** : `https://api_key:api_secret@hub.testingbot.com/wd/hub` ![katalon studio with TestingBot](https://testingbot.com/assets/support/katalon-bc59f7c086cd9d849763e6b6846c8a5bfbe38ec5804b8905fae42da5ba97fe6f.png) Fill in the `browserName`, `version` and `platform` or any [other desired capabilities](https://testingbot.com/support/web-automate/selenium/test-options) you want to use. ## Run Katalon Studio test on TestingBot To run the recorded test on TestingBot, click **Action \> Run** and then **Remote**. ![run Katalone test on TestingBot](https://testingbot.com/assets/support/katalon-remote-d0844ef224eb68c19a3cbd28c81dac618fa4d77587a3ff3ff70593e1dfa59cf4.png) ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/python ### Python Examples: - [PyTest](https://testingbot.com/support/web-automate/selenium/python/pytest) - [Behave](https://testingbot.com/support/web-automate/selenium/python/behave) - [Lettuce](https://testingbot.com/support/web-automate/selenium/python/lettuce) - [PyUnit](https://testingbot.com/support/web-automate/selenium/python/pyunit) - [Helium](https://testingbot.com/support/web-automate/selenium/python/helium) - [Pylenium](https://testingbot.com/support/web-automate/selenium/python/pylenium) - [SeleniumBase](https://testingbot.com/support/web-automate/selenium/python/seleniumbase) - [Robot Framework](https://testingbot.com/support/web-automate/selenium/robotframework) # Python Automated Testing Before you can do any kind of integration testing with Python, let's first start with making sure Python is available on your system. **For Windows:** - Download the latest Python installer for Windows: [Python Releases for Windows](https://www.python.org/downloads/windows/) - Run the installer and follow the setup wizard to install Python. **For Linux/macOS:** - Run `python3 --version` to see which Python version is currently installed. Make sure it is Python 3.8 or above. - macOS and most Linux distros come with Python pre-installed. You may need to install `python3` separately on some systems. ## Installation [![Video tutorial on how to run a Python Automated Test on TestingBot](https://testingbot.com/assets/videos/python-automated-c0afd41f66658429632dbc218f3948be42b059eeae854abf1093d20487431dee.webp)](https://testingbot.com#) about:blank With TestingBot you can easily run your automated tests with [any Python testing framework](https://testingbot.com#frameworks), here's a simple example: ```bash pip install selenium pip install testingbotclient ``` ```python import unittest from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.chrome.options import Options as ChromeOptions from testingbotclient import TestingBotClient class TestTestingBotClient(unittest.TestCase): def setUp(self): options = ChromeOptions() options.set_capability('platformName', 'WIN11') options.set_capability('browserVersion', 'latest') options.set_capability('tb:options', { 'key': 'API_KEY', 'secret': 'API_SECRET', 'name': self._testMethodName }) self.driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options ) def test_google_example(self): self.driver.get("https://www.google.com") if "Google" not in self.driver.title: raise Exception("Unable to load Google page!") elem = self.driver.find_element(By.NAME, "q") elem.send_keys("TestingBot") elem.submit() def tearDown(self): if hasattr(self._outcome, 'errors'): # Python <= 3.10 result = self.defaultTestResult() self._feedErrorsToResult(result, self._outcome.errors) passed = not (result.errors or result.failures) else: # Python 3.11+ result = self._outcome.result passed = all(test is not self for test, _ in result.errors + result.failures) session_id = self.driver.session_id try: tb_client = TestingBotClient( 'API_KEY', 'API_SECRET' ) tb_client.tests.update_test(session_id, passed=passed) finally: self.driver.quit() if __name__ == ' __main__': unittest.main() ``` This test will start a Chrome browser on Windows in our cloud, go to Google, and search for TestingBot. Once the test has finished, the `tb_client` library will send back the test's success state to TestingBot, so you have a nice overview of your tests by name and success state in our member dashboard. Make sure to always stop your test (`driver.quit()`), otherwise it will continue running, leading to a timeout. ### Configuring capabilities To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before:** ```python driver = webdriver.Firefox() ``` **After:** ```python from selenium import webdriver from selenium.webdriver.chrome.options import Options as ChromeOptions options = ChromeOptions() options.set_capability('platformName', 'WIN11') options.set_capability('browserVersion', 'latest') options.set_capability('tb:options', { 'key': 'API_KEY', 'secret': 'API_SECRET' }) driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options ) ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a Python test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```python import unittest from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.chrome.options import Options as ChromeOptions from testingbotclient import TestingBotClient class TestTestingBotClient(unittest.TestCase): def setUp(self): options = ChromeOptions() options.set_capability('platformName', 'WIN11') options.set_capability('browserVersion', 'latest') options.set_capability('tb:options', { 'key': 'API_KEY', 'secret': 'API_SECRET', 'name': self._testMethodName }) # Point to the tunnel running on localhost:4445 self.driver = webdriver.Remote( command_executor='http://localhost:4445/wd/hub', options=options ) def test_google_example(self): self.driver.get("https://www.google.com") if "Google" not in self.driver.title: raise Exception("Unable to load Google page!") elem = self.driver.find_element(By.NAME, "q") elem.send_keys("TestingBot") elem.submit() def tearDown(self): if hasattr(self._outcome, 'errors'): # Python <= 3.10 result = self.defaultTestResult() self._feedErrorsToResult(result, self._outcome.errors) passed = not (result.errors or result.failures) else: # Python 3.11+ result = self._outcome.result passed = all(test is not self for test, _ in result.errors + result.failures) session_id = self.driver.session_id try: tb_client = TestingBotClient( 'API_KEY', 'API_SECRET' ) tb_client.tests.update_test(session_id, passed=passed) finally: self.driver.quit() if __name__ == ' __main__': unittest.main() ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. To run tests in parallel, we recommend using pytest with pytest-xdist, which makes it very easy to run multiple Python tests simultaneously: ```bash pip install pytest pytest-xdist ``` ```bash pytest -n 4 # run tests on 4 parallel workers ``` ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. You can use our [Python API client](https://github.com/testingbot/testingbotclient) to report back test results. ```python from testingbotclient import TestingBotClient client = TestingBotClient('API_KEY', 'API_SECRET') client.tests.update_test( driver.session_id, name='My Test', passed=True, build='build-123' ) ``` ## Other Python Framework examples - [PyTest](https://testingbot.com/support/web-automate/selenium/python/pytest) PyTest makes it easy to run Selenium tests with Python. - [Behave](https://testingbot.com/support/web-automate/selenium/python/behave) Behave is behaviour-driven development, Python style. - [Lettuce](https://testingbot.com/support/web-automate/selenium/python/lettuce) Lettuce is a Python BDD plugin based on Ruby's Cucumber, offering Gherkin stories. - [PyUnit](https://testingbot.com/support/web-automate/selenium/python/pyunit) PyUnit is the standard unit testing framework module for Python, described as a Python version of JUnit. - [Helium](https://testingbot.com/support/web-automate/selenium/python/helium) Helium is a tool that makes it easy to test websites and automate browsers. - [Pylenium](https://testingbot.com/support/web-automate/selenium/python/pylenium) Pylenium brings the best of Selenium, Cypress and Python into one package. - [SeleniumBase](https://testingbot.com/support/web-automate/selenium/python/seleniumbase) SeleniumBase is a Python package that simplifies Selenium testing. - [Robot Framework](https://testingbot.com/support/web-automate/selenium/robotframework) Robot Framework is a generic automation framework for acceptance testing. See our [Selenium Python tutorial](https://testingbot.com/resources/articles/python-selenium-web-automation-test) for a first test written from scratch, and [handling cookies in Selenium WebDriver](https://testingbot.com/resources/articles/handling-cookies-with-selenium) for reading, setting and clearing cookies mid-session. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/python/pyunit # PyUnit Automated Testing See our [PyUnit example repository](https://github.com/testingbot/testingbot_pyunit) for a simple example on how to run PyUnit tests on TestingBot. Let's start with making sure Python is available on your system. **For Windows:** - Download the latest Python installer for Windows: [Python Releases for Windows](https://www.python.org/downloads/windows/) - Run the installer and follow the setup wizard to install Python. **For Linux/macOS:** - Run `python3 --version` to see which Python version is currently installed. Make sure it is Python 3.8 or above. - macOS and most Linux distros come with Python pre-installed. You may need to install `python3` separately on some systems. ## Installation First, make sure you have installed the correct Python packages: ```bash pip install selenium pip install testingbotclient ``` You are now ready to start testing with our Selenium grid. ### Example code ```python import unittest from selenium import webdriver from selenium.webdriver.chrome.options import Options as ChromeOptions from testingbotclient import TestingBotClient class TestGoogleSearch(unittest.TestCase): def setUp(self): options = ChromeOptions() options.set_capability('platformName', 'WIN11') options.set_capability('browserVersion', 'latest') options.set_capability('tb:options', { 'key': 'API_KEY', 'secret': 'API_SECRET', 'name': 'Testing Selenium with PyUnit' }) self.driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options ) self.driver.implicitly_wait(30) def test_google(self): self.driver.get('https://www.google.com') self.assertIn("Google", self.driver.title) def tearDown(self): if hasattr(self._outcome, 'errors'): # Python <= 3.10 result = self.defaultTestResult() self._feedErrorsToResult(result, self._outcome.errors) passed = not (result.errors or result.failures) else: # Python 3.11+ result = self._outcome.result passed = all(test is not self for test, _ in result.errors + result.failures) session_id = self.driver.session_id try: tb_client = TestingBotClient( 'API_KEY', 'API_SECRET' ) tb_client.tests.update_test(session_id, passed=passed) finally: self.driver.quit() if __name__ == ' __main__': unittest.main() ``` ### Configuring capabilities To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before:** ```python driver = webdriver.Firefox() ``` **After:** ```python from selenium import webdriver from selenium.webdriver.chrome.options import Options as ChromeOptions options = ChromeOptions() options.set_capability('platformName', 'WIN11') options.set_capability('browserVersion', 'latest') options.set_capability('tb:options', { 'key': 'API_KEY', 'secret': 'API_SECRET' }) driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options ) ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a PyUnit test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```python import unittest from selenium import webdriver from selenium.webdriver.chrome.options import Options as ChromeOptions from testingbotclient import TestingBotClient class TestGoogleSearch(unittest.TestCase): def setUp(self): options = ChromeOptions() options.set_capability('platformName', 'WIN11') options.set_capability('browserVersion', 'latest') options.set_capability('tb:options', { 'key': 'API_KEY', 'secret': 'API_SECRET', 'name': 'Testing Selenium with PyUnit' }) # Point to the tunnel running on localhost:4445 self.driver = webdriver.Remote( command_executor='http://localhost:4445/wd/hub', options=options ) self.driver.implicitly_wait(30) def test_google(self): self.driver.get('https://www.google.com') self.assertIn("Google", self.driver.title) def tearDown(self): if hasattr(self._outcome, 'errors'): # Python <= 3.10 result = self.defaultTestResult() self._feedErrorsToResult(result, self._outcome.errors) passed = not (result.errors or result.failures) else: # Python 3.11+ result = self._outcome.result passed = all(test is not self for test, _ in result.errors + result.failures) session_id = self.driver.session_id try: tb_client = TestingBotClient( 'API_KEY', 'API_SECRET' ) tb_client.tests.update_test(session_id, passed=passed) finally: self.driver.quit() if __name__ == ' __main__': unittest.main() ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. To run tests in parallel, we recommend using pytest with pytest-xdist, which makes it very easy to run multiple Python tests simultaneously: ```bash pip install pytest pytest-xdist ``` ```bash pytest -n 4 # run tests on 4 parallel workers ``` ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. You can use our [Python API client](https://github.com/testingbot/testingbotclient) to report back test results. ```python from testingbotclient import TestingBotClient client = TestingBotClient('API_KEY', 'API_SECRET') client.tests.update_test( driver.session_id, name='My Test', passed=True, build='build-123' ) ``` ## Other Python Framework examples - [PyTest](https://testingbot.com/support/web-automate/selenium/python/pytest) PyTest makes it easy to run Selenium tests with Python. - [Behave](https://testingbot.com/support/web-automate/selenium/python/behave) Behave is behaviour-driven development, Python style. - [Lettuce](https://testingbot.com/support/web-automate/selenium/python/lettuce) Lettuce is a Python BDD plugin based on Ruby's Cucumber, offering Gherkin stories. - [PyUnit](https://testingbot.com/support/web-automate/selenium/python/pyunit) PyUnit is the standard unit testing framework module for Python, described as a Python version of JUnit. - [Helium](https://testingbot.com/support/web-automate/selenium/python/helium) Helium is a tool that makes it easy to test websites and automate browsers. - [Pylenium](https://testingbot.com/support/web-automate/selenium/python/pylenium) Pylenium brings the best of Selenium, Cypress and Python into one package. - [SeleniumBase](https://testingbot.com/support/web-automate/selenium/python/seleniumbase) SeleniumBase is a Python package that simplifies Selenium testing. - [Robot Framework](https://testingbot.com/support/web-automate/selenium/robotframework) Robot Framework is a generic automation framework for acceptance testing. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/python/lettuce # BDD with Python, Lettuce and WebDriver **Note:** Lettuce is no longer actively maintained. Consider using [Behave](https://testingbot.com/support/web-automate/selenium/python/behave) or [pytest-bdd](https://pytest-bdd.readthedocs.io/) for new projects. Lettuce is a Python BDD plugin based on Ruby's Cucumber, offering Gherkin stories. To get started, make sure you have installed these packages with `pip`: ```bash pip install lettuce lettuce_webdriver nose ``` `lettuce` is a Python 2 only project and is no longer maintained. The packages _install_ on modern Python, but `lettuce` fails to **import/run** on Python 3 (it raises a `SyntaxError` from Python 2 `print` statements and the removed `ur''` literal), so the workflow on this page only works on a legacy Python 2.7 interpreter. For new projects on Python 3, use [Behave](https://testingbot.com/support/web-automate/selenium/python/behave) or [pytest-bdd](https://pytest-bdd.readthedocs.io/) instead. You are now ready to create your first story and run it on our Selenium grid. Run the test with: ```bash lettuce ``` ## Example feature (features/google.feature) ```yaml Feature: Go to google Scenario: Visit Google Given I go to "https://www.google.com/" When I fill in field with class "gsfi" with "testingbot" Then I should see "testingbot.com" within 2 seconds ``` ## Example steps (features/steps.py) ```python from lettuce import * from lettuce_webdriver.util import assert_false from lettuce_webdriver.util import AssertContextManager from selenium.webdriver.common.by import By def find_field_by_class(browser, attribute): xpath = "//input[@class='%s']" % attribute elems = browser.find_elements(By.XPATH, xpath) return elems[0] if elems else False @step('I fill in field with class "(.*?)" with "(.*?)"') def fill_in_textfield_by_class(step, field_name, value): with AssertContextManager(step): text_field = find_field_by_class(world.browser, field_name) text_field.clear() text_field.send_keys(value) ``` ## Example terrain.py ```python from lettuce import after, before, world from selenium import webdriver from selenium.webdriver.firefox.options import Options import lettuce_webdriver.webdriver @before.all def setup_browser(): options = Options() options.browser_version = 'latest' options.platform_name = 'WIN11' options.set_capability('tb:options', { 'key': 'API_KEY', 'secret': 'API_SECRET', 'name': 'Testing Selenium with Lettuce' }) world.browser = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options ) @after.all def teardown_browser(total): world.browser.quit() ``` ## Configuring capabilities To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before:** ```python driver = webdriver.Firefox() ``` **After:** ```python from selenium.webdriver.firefox.options import Options options = Options() # set capabilities from picker below driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options ) ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a Lettuce Python test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ### Example terrain.py ```python from lettuce import before, world from selenium import webdriver from selenium.webdriver.firefox.options import Options import lettuce_webdriver.webdriver @before.all def setup_browser(): options = Options() options.browser_version = 'latest' options.platform_name = 'WIN10' options.set_capability('tb:options', { 'key': 'API_KEY', 'secret': 'API_SECRET', 'name': 'Testing Selenium with Lettuce' }) world.browser = webdriver.Remote( command_executor='http://localhost:4445/wd/hub', options=options ) ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. To run tests in parallel, we recommend using pytest with pytest-xdist, which makes it easy to run multiple Python tests simultaneously: ```bash pip install pytest pytest-xdist ``` ```bash pytest -n ``` ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. You can use our [Python API client](https://github.com/testingbot/testingbotclient) to report back test results. ```python from testingbotclient import TestingBotClient tb = TestingBotClient("key", "secret") tb.tests.update_test(world.browser.session_id, name="My Test", passed=True, build="build-123") ``` ## Other Python Framework examples - [PyTest](https://testingbot.com/support/web-automate/selenium/python/pytest) PyTest makes it easy to run Selenium tests with Python. - [Behave](https://testingbot.com/support/web-automate/selenium/python/behave) Behave is behaviour-driven development, Python style. - [PyUnit](https://testingbot.com/support/web-automate/selenium/python/pyunit) PyUnit is the standard unit testing framework module for Python, described as a Python version of JUnit. - [Helium](https://testingbot.com/support/web-automate/selenium/python/helium) Helium is a tool that makes it easy to test websites and automate browsers. - [Pylenium](https://testingbot.com/support/web-automate/selenium/python/pylenium) Pylenium is a Python package, its mission: "Bring the best of Selenium, Cypress and Python into one package." ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/python/behave # BDD with Python, Behave and WebDriver See our [Behave example repository](https://github.com/testingbot/python-behave-example) for a simple example on how to run Behave tests in parallel on TestingBot. Behave is a Python BDD (Behavior Driven Development) framework which makes it easy to write tests (bdd tests) in a natural language style. To get started, make sure you have installed Behave, Selenium and the TestingBot client: ```bash pip install behave selenium testingbotclient ``` You are now ready to create your first story and run it on our Selenium grid. Run the test with: ```bash behave ``` ## Example feature (features/google.feature) This feature file needs to be added to the features directory. ```yaml Feature: testing google Scenario: visit google and check When we visit google Then it should have a title "Google" ``` ## Example steps (features/steps/steps.py) ```python from behave import when, then @when('we visit google') def step_visit_google(context): context.browser.get('https://www.google.com') @then('it should have a title "Google"') def step_check_title(context): assert context.browser.title == "Google" ``` ## Example features/environment.py ```python from selenium import webdriver from selenium.webdriver.firefox.options import Options def before_all(context): options = Options() options.set_capability('browserVersion', 'latest') options.set_capability('platformName', 'WIN11') options.set_capability('tb:options', { 'name': 'Testing Selenium with Behave', 'key': 'key', 'secret': 'secret' }) context.browser = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options ) def after_all(context): context.browser.quit() ``` ## Configuring capabilities To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before:** ```python driver = webdriver.Firefox() ``` **After:** ```python from selenium.webdriver.firefox.options import Options options = Options() options.set_capability('browserVersion', 'latest') options.set_capability('platformName', 'WIN10') options.set_capability('tb:options', { 'key': 'key', 'secret': 'secret' }) driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options ) ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a Behave WebDriver test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ### Example features/environment.py ```python from selenium import webdriver from selenium.webdriver.firefox.options import Options def before_all(context): options = Options() options.set_capability('browserVersion', 'latest') options.set_capability('platformName', 'WIN10') options.set_capability('tb:options', { 'name': 'Testing Selenium with Behave', 'key': 'key', 'secret': 'secret' }) context.browser = webdriver.Remote( command_executor='http://localhost:4445/wd/hub', options=options ) def after_all(context): context.browser.quit() ``` ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. You can use our [Python API client](https://github.com/testingbot/testingbotclient) to report back test results. ```python from testingbotclient import TestingBotClient tb = TestingBotClient("key", "secret") tb.tests.update_test(driver.session_id, name="My Test", passed=True, build="build-123") ``` ## Other Python Framework examples - [PyTest](https://testingbot.com/support/web-automate/selenium/python/pytest) PyTest makes it easy to run Selenium tests with Python. - [Behave](https://testingbot.com/support/web-automate/selenium/python/behave) Behave is behaviour-driven development, Python style. - [Lettuce](https://testingbot.com/support/web-automate/selenium/python/lettuce) Lettuce is a Python BDD plugin based on Ruby's Cucumber, offering Gherkin stories. - [PyUnit](https://testingbot.com/support/web-automate/selenium/python/pyunit) PyUnit is the standard unit testing framework module for Python, described as a Python version of JUnit. - [Helium](https://testingbot.com/support/web-automate/selenium/python/helium) Helium is a tool that makes it easy to test websites and automate browsers. - [Pylenium](https://testingbot.com/support/web-automate/selenium/python/pylenium) Pylenium is a Python package, its mission: "Bring the best of Selenium, Cypress and Python into one package." ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/python/helium # Helium Automated Browser Testing Helium is a tool that makes it easy to test websites and automate browsers. It's designed to be very simple to use and integrate. Helium can be used as a Java or Python library and integrates with the Selenium Framework. ## Setting up Helium There's a very good tutorial on how to set up Helium on the [Helium Repository](https://github.com/mherrmann/selenium-python-helium) page. Install Helium (it pulls in Selenium as a dependency): ```bash pip install helium selenium ``` Once you've got everything set up, it's easy to run your first test with Helium on TestingBot: ## Running your first test Below is an example on how to run a simple test on Firefox. When the test has finished, the test name and success state is sent to TestingBot so you can see the test success/failures in our dashboard. To send the test success state back to us, please install our Python plugin: ```bash pip install testingbotclient ``` **Python Example:** ```python import unittest from selenium import webdriver from selenium.webdriver.firefox.options import Options from helium import * from testingbotclient import TestingBotClient class HeliumOnTestingBot(unittest.TestCase): def setUp(self): options = Options() options.browser_version = 'latest' options.platform_name = 'WIN11' options.set_capability('tb:options', { 'key': 'API_KEY', 'secret': 'API_SECRET', 'name': 'My First Test' }) self.driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options ) set_driver(self.driver) def test_tb(self): go_to('https://testingbot.com') click('Log in') write('my@email.com', into='Email') write('myPassword', into='Password') click(Button('Sign in')) def tearDown(self): if hasattr(self._outcome, 'errors'): # Python <= 3.10 result = self.defaultTestResult() self._feedErrorsToResult(result, self._outcome.errors) status = not (result.errors or result.failures) else: # Python 3.11+ result = self._outcome.result status = all(test is not self for test, _ in result.errors + result.failures) session_id = self.driver.session_id self.driver.quit() tb_client = TestingBotClient('API_KEY', 'API_SECRET') tb_client.tests.update_test(session_id, name=self._testMethodName, passed=status) if __name__ == ' __main__': unittest.main() ``` **Java Example:** ```java import org.junit.After; import org.junit.Before; import org.junit.Test; import org.openqa.selenium.WebDriver; import org.openqa.selenium.firefox.FirefoxOptions; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URL; import java.util.HashMap; import static org.junit.Assert.assertEquals; import static com.heliumhq.API.*; public class WebDriverTest { private WebDriver driver; @Before public void setUp() throws Exception { FirefoxOptions options = new FirefoxOptions(); options.setPlatformName("WIN11"); options.setBrowserVersion("latest"); HashMap tbOptions = new HashMap<>(); tbOptions.put("key", "API_KEY"); tbOptions.put("secret", "API_SECRET"); tbOptions.put("name", "My First Test"); options.setCapability("tb:options", tbOptions); this.driver = new RemoteWebDriver( new URL("https://hub.testingbot.com/wd/hub"), options); setDriver(this.driver); } @Test public void testWithTestingBot() throws Exception { goTo("https://testingbot.com"); click("Log in"); assertEquals("Online Selenium Testing - Log in to manage your unit tests.", getDriver().getTitle()); write("user12345", into("Email:")); write("user12345", into("Password:")); click(Button("Sign in")); } @After public void tearDown() throws Exception { driver.quit(); } } ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. **Before** ```python driver = webdriver.Firefox() ``` **After** ```python from selenium.webdriver.firefox.options import Options options = Options() # set capabilities from picker below driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options) ``` ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## More Information More information regarding Helium is available on the [Helium Repository](https://github.com/mherrmann/selenium-python-helium). ## Other Python Framework examples - [PyTest](https://testingbot.com/support/web-automate/selenium/python/pytest) PyTest makes it easy to run Selenium tests with Python. - [Behave](https://testingbot.com/support/web-automate/selenium/python/behave) Behave is behaviour-driven development, Python style. - [Lettuce](https://testingbot.com/support/web-automate/selenium/python/lettuce) Lettuce is a Python BDD plugin based on Ruby's Cucumber, offering Gherkin stories. - [PyUnit](https://testingbot.com/support/web-automate/selenium/python/pyunit) PyUnit is the standard unit testing framework module for Python, described as a Python version of JUnit. - [Helium](https://testingbot.com/support/web-automate/selenium/python/helium) Helium is a tool that makes it easy to test websites and automate browsers. - [Pylenium](https://testingbot.com/support/web-automate/selenium/python/pylenium) Pylenium is a Python package, its mission: "Bring the best of Selenium, Cypress and Python into one package." - [SeleniumBase](https://testingbot.com/support/web-automate/selenium/python/seleniumbase) SeleniumBase is a Python package that simplifies Selenium testing. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/python/pytest # PyTest Automated Testing See our [PyTest Selenium Example](https://github.com/testingbot/Python-PyTest-Selenium) for a simple example on how to run PyTest tests on TestingBot. Let's start with making sure Python is available on your system. **For Windows:** - Download the latest Python installer for Windows: [Python Releases for Windows](https://www.python.org/downloads/windows/) - Run the installer and follow the setup wizard to install Python. **For Linux/macOS:** - Run `python3 --version` to see which Python version is currently installed. Make sure it is Python 3.8 or above. - macOS and most Linux distros come with Python pre-installed. You may need to install `python3` separately on some systems. ## Installation Install `pytest` and `selenium`: ```bash pip install pytest selenium ``` You are now ready to start testing with our Selenium grid. ### Setup credentials Set your TestingBot credentials as environment variables. The fixture below reads them when constructing the remote driver: ```bash export TESTINGBOT_KEY=API_KEY export TESTINGBOT_SECRET=API_SECRET ``` ### conftest.py Create a `conftest.py` file that defines a `driver` fixture pointing at the TestingBot grid: ```python import os import pytest from selenium import webdriver @pytest.fixture def driver(request): options = webdriver.ChromeOptions() options.browser_version = 'latest' options.platform_name = 'WIN11' options.set_capability('tb:options', { 'key': os.environ['TESTINGBOT_KEY'], 'secret': os.environ['TESTINGBOT_SECRET'], 'name': request.node.name, }) driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options, ) yield driver driver.quit() ``` ### Example code Save the example in a file called `test_sample.py`. (Important: the filename should start with `test_`) ```python def test_sample(driver): driver.get('https://google.com/ncr') assert "Google" == driver.title ``` Run the test: ```bash pytest test_sample.py ``` This will run the test on the latest version of Chrome on Windows 11. Update the capabilities in the fixture (or parametrize the fixture) to target other browsers and platforms. ### Configuring capabilities To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before:** ```python driver = webdriver.Chrome() ``` **After:** ```python driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options, ) ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a PyTest WebDriver test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret -i pyTunnel ``` 2. In your `conftest.py` fixture, point the driver at the local tunnel and set `tunnelIdentifier` inside `tb:options`: ```python options.set_capability('tb:options', { 'key': os.environ['TESTINGBOT_KEY'], 'secret': os.environ['TESTINGBOT_SECRET'], 'tunnelIdentifier': 'pyTunnel', }) driver = webdriver.Remote( command_executor='http://localhost:4445/wd/hub', options=options, ) ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. You can use pytest-xdist's `-n` option to run multiple tests at the same time: ```bash pip install pytest-xdist ``` ```bash pytest -n 4 test_sample.py ``` ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. You can use our [Python API client](https://github.com/testingbot/testingbotclient) to report back test results. Install it with `pip install testingbotclient`: ```python from testingbotclient import TestingBotClient client = TestingBotClient('API_KEY', 'API_SECRET') client.tests.update_test( driver.session_id, name='My Test', passed=True, build='build-123' ) ``` ## Other Python Framework examples - [PyTest](https://testingbot.com/support/web-automate/selenium/python/pytest) PyTest makes it easy to run Selenium tests with Python. - [Behave](https://testingbot.com/support/web-automate/selenium/python/behave) Behave is behaviour-driven development, Python style. - [Lettuce](https://testingbot.com/support/web-automate/selenium/python/lettuce) Lettuce is a Python BDD plugin based on Ruby's Cucumber, offering Gherkin stories. - [PyUnit](https://testingbot.com/support/web-automate/selenium/python/pyunit) PyUnit is the standard unit testing framework module for Python, described as a Python version of JUnit. - [Helium](https://testingbot.com/support/web-automate/selenium/python/helium) Helium is a tool that makes it easy to test websites and automate browsers. - [Pylenium](https://testingbot.com/support/web-automate/selenium/python/pylenium) Pylenium brings the best of Selenium, Cypress and Python into one package. - [Robot Framework](https://testingbot.com/support/web-automate/selenium/robotframework) Robot Framework is a generic automation framework for acceptance testing. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/python/pylenium # Pylenium Pylenium is a Python package, its mission: "Bring the best of Selenium, Cypress and Python into one package." To get started, please make sure you have **python3** installed. Then install the package and scaffold the project: ```bash pip install pyleniumio pylenium init ``` `pylenium init` creates a `pylenium.json` config file and a `conftest.py` in your project. ## Example test Pylenium tests use the `py` fixture that ships with the package. Save the example as `test_sample.py`: ```python def test_sample(py): py.visit("https://www.google.com") py.get("[name='q']").type("TestingBot") assert "Google" in py.title() ``` Run it with pytest: ```bash pytest test_sample.py ``` ## Specify Browsers & Devices Point Pylenium at the TestingBot grid by editing `pylenium.json`. Set `driver.remote_url` to the TestingBot hub and put the W3C capabilities under `driver.capabilities`: ```json { "driver": { "browser": "chrome", "remote_url": "https://API_KEY:API_SECRET@hub.testingbot.com/wd/hub", "capabilities": { "browserVersion": "latest", "platformName": "WIN11", "tb:options": { "name": "Pylenium Test" } } } } ``` Run the same test command, Pylenium now provisions the browser on TestingBot: ```bash pytest test_sample.py ``` To use a different browser, version or platform on the fly, override capabilities from the CLI: ```bash pytest --remote_url='https://API_KEY:API_SECRET@hub.testingbot.com/wd/hub' ``` Pick a browser/version/platform combination below, the snippet shows the JSON fragment to drop under `driver.capabilities` in `pylenium.json`. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a Pylenium Python test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back. Point Pylenium at the local tunnel URL in `pylenium.json`: ```json { "driver": { "browser": "chrome", "remote_url": "http://localhost:4445/wd/hub", "capabilities": { "browserVersion": "latest", "platformName": "WIN11", "tb:options": { "name": "Pylenium Tunnel Test" } } } } ``` ## Other Options We offer many other [test options](https://testingbot.com/support/web-automate/selenium/test-options), for example: disable video recording, specifying a custom Firefox Profile, loading Chrome/Firefox/Safari extensions, running an executable before your test starts, uploading files, ... See our [list of test options](https://testingbot.com/support/web-automate/selenium/test-options) for a full list of options to customize your tests. ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. To run tests in parallel, we recommend using pytest with pytest-xdist, which makes it very easy to run multiple Python tests simultaneously: ```bash pip install pytest pytest-xdist ``` ```bash pytest -n 4 # run tests on 4 parallel workers ``` ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. You can use our [Python API client](https://github.com/testingbot/testingbotclient) to report back test results. Pylenium exposes the underlying Selenium WebDriver via `py.webdriver`, so the session id is available as `py.webdriver.session_id`: ```python from testingbotclient import TestingBotClient def test_sample(py): py.visit("https://www.google.com") passed = "Google" in py.title() client = TestingBotClient('API_KEY', 'API_SECRET') client.tests.update_test( py.webdriver.session_id, name='My Test', passed=passed, build='build-123' ) assert passed ``` ## Other Python Framework examples - [PyTest](https://testingbot.com/support/web-automate/selenium/python/pytest) PyTest makes it easy to run Selenium tests with Python. - [Behave](https://testingbot.com/support/web-automate/selenium/python/behave) Behave is behaviour-driven development, Python style. - [Lettuce](https://testingbot.com/support/web-automate/selenium/python/lettuce) Lettuce is a Python BDD plugin based on Ruby's Cucumber, offering Gherkin stories. - [PyUnit](https://testingbot.com/support/web-automate/selenium/python/pyunit) PyUnit is the standard unit testing framework module for Python, described as a Python version of JUnit. - [Pylenium](https://testingbot.com/support/web-automate/selenium/python/pylenium) Pylenium brings the best of Selenium, Cypress and Python into one package. - [Helium](https://testingbot.com/support/web-automate/selenium/python/helium) Helium is a tool that makes it easy to test websites and automate browsers. - [Robot Framework](https://testingbot.com/support/web-automate/selenium/robotframework) Robot Framework is a generic automation framework for acceptance testing. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/python/seleniumbase # SeleniumBase Browser Automation Framework SeleniumBase is a Python-based framework that extends Selenium WebDriver, making it easier to write and run automated web tests. SeleniumBase includes many built-in features such as visual testing, screenshot comparisons, and handling of dynamic web content. One of the key highlights of SeleniumBase is its ability to integrate smoothly with popular Python testing frameworks like pytest. This enables parallel test execution, better test organization and improved scalability for larger projects. ## Setting up SeleniumBase To install SeleniumBase, you can use pip to install the package: ```bash pip install seleniumbase ``` ## Running your first test Once you've installed SeleniumBase, you can create your first test that will connect to a remote TestingBot browser. **Python Example:** ```python from seleniumbase import BaseCase class DemoSiteTests(BaseCase): def test_demo_site(self): # Open a web page in the active browser window self.open("https://seleniumbase.io/demo_page") # Assert the title of the current web page self.assert_title("Web Testing Page") # Assert that an element is visible on the page self.assert_element("tbody#tbodyId") # Assert that a text substring appears in an element self.assert_text("Demo Page", "h1") if __name__ == " __main__": BaseCase.main( __name__ , __file__ ) ``` You can now run this script on a TestingBot remote browser: ```bash pytest test_demo_site.py --server=API_KEY:API_SECRET@hub.testingbot.com --port=443 --protocol=https --cap_file=capabilities/cap_file_TB.py ``` The capabilities are passed as a file, in this case an example `cap_file_TB.py` could look like this: ```python capabilities = { "browserName": "chrome", "browserVersion": "latest", "platformName": "WIN11", "tb:options": { "name": "SeleniumBase Test" } } ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform you want to run your test on, you need to specify the browsername, version, OS and other optional options in a capabilities file. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a SeleniumBase test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test command to point to your tunnel's IP address: ```bash pytest test_demo_site.py --server=localhost --port=4445 --protocol=http --cap_file=capabilities/cap_file_TB.py ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. SeleniumBase supports parallel test execution using pytest-xdist: ```bash pip install pytest-xdist ``` ```bash pytest -n 4 --server=API_KEY:API_SECRET@hub.testingbot.com --port=443 --protocol=https --cap_file=capabilities/cap_file_TB.py ``` ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. You can use our [Python API client](https://github.com/testingbot/testingbotclient) to report back test results. Install it with: ```bash pip install testingbotclient ``` Call it from within your `BaseCase` test (for example in `tearDown`), where the WebDriver is available as `self.driver`: ```python from testingbotclient import TestingBotClient client = TestingBotClient('API_KEY', 'API_SECRET') client.tests.update_test( self.driver.session_id, name='My Test', passed=True, build='build-123' ) ``` ## Other Python Framework examples - [PyTest](https://testingbot.com/support/web-automate/selenium/python/pytest) PyTest makes it easy to run Selenium tests with Python. - [Behave](https://testingbot.com/support/web-automate/selenium/python/behave) Behave is behaviour-driven development, Python style. - [Lettuce](https://testingbot.com/support/web-automate/selenium/python/lettuce) Lettuce is a Python BDD plugin based on Ruby's Cucumber, offering Gherkin stories. - [PyUnit](https://testingbot.com/support/web-automate/selenium/python/pyunit) PyUnit is the standard unit testing framework module for Python, described as a Python version of JUnit. - [Helium](https://testingbot.com/support/web-automate/selenium/python/helium) Helium is a tool that makes it easy to test websites and automate browsers. - [Pylenium](https://testingbot.com/support/web-automate/selenium/python/pylenium) Pylenium brings the best of Selenium, Cypress and Python into one package. - [SeleniumBase](https://testingbot.com/support/web-automate/selenium/python/seleniumbase) SeleniumBase is a Python package that simplifies Selenium testing. - [Robot Framework](https://testingbot.com/support/web-automate/selenium/robotframework) Robot Framework is a generic automation framework for acceptance testing. ## More Information More information regarding SeleniumBase is available on the [SeleniumBase documentation](https://seleniumbase.io/seleniumbase/utilities/selenium_grid/ReadMe/). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/ruby ### Ruby Examples: - [Capybara](https://testingbot.com/support/web-automate/selenium/ruby/capybara) - [Cucumber](https://testingbot.com/support/web-automate/selenium/ruby/cucumber) - [RSpec](https://testingbot.com/support/web-automate/selenium/ruby/rspec) - [Test::Unit](https://testingbot.com/support/web-automate/selenium/ruby/testunit) - [Minitest](https://testingbot.com/support/web-automate/selenium/ruby/minitest) - [Watir](https://testingbot.com/support/web-automate/selenium/ruby/watir) # Ruby Automated Testing Let's start with making sure Ruby is available on your system. **For Windows:** - You can download Ruby for Windows from [RubyInstaller](https://rubyinstaller.org/) - Run the installer and follow the setup wizard to install Ruby. **For Linux/macOS:** Download [RVM](https://rvm.io/) and install Ruby: ```bash curl -L https://get.rvm.io | bash -s stable --ruby ``` ## Installation [![Video tutorial on How to run Ruby Automated Test on TestingBot](https://testingbot.com/assets/videos/ruby-automated-bf8e6bafdf31a6b815fdafc10a58e85d93b99b8cd4e788fc3ecb6d652f495a3e.webp)](https://testingbot.com#) about:blank With TestingBot you can easily run your automated tests with [any Ruby test framework](https://testingbot.com#frameworks), here's a simple example without any assertions: ```bash gem install selenium-webdriver ``` ```ruby #!/usr/bin/env ruby require 'selenium-webdriver' options = Selenium::WebDriver::Chrome::Options.new options.add_option('platformName', 'WIN11') options.add_option('browserVersion', 'latest') options.add_option('tb:options', { 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'My First Test' }) driver = Selenium::WebDriver.for( :remote, url: 'https://hub.testingbot.com/wd/hub', options: options ) driver.navigate.to 'https://www.google.com/ncr' element = driver.find_element(:name, 'q') element.send_keys 'TestingBot', :return puts driver.title driver.quit ``` This test will start the latest version of Chrome on Windows in our cloud, go to Google, and search for TestingBot. It will then output the title of the page. Make sure to always stop your test (`driver.quit`), otherwise it will continue running, leading to a timeout. ### Configuring capabilities To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before:** ```ruby driver = Selenium::WebDriver.for :chrome ``` **After:** ```ruby options = Selenium::WebDriver::Chrome::Options.new options.add_option('platformName', 'WIN11') options.add_option('browserVersion', 'latest') options.add_option('tb:options', { 'key' => 'API_KEY', 'secret' => 'API_SECRET' }) driver = Selenium::WebDriver.for(:remote, url: 'https://hub.testingbot.com/wd/hub', options: options ) ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ```ruby options = Selenium::WebDriver::Chrome::Options.new options.add_option('platformName', 'WIN11') options.add_option('browserVersion', 'latest') options.add_option('tb:options', { 'key' => 'API_KEY', 'secret' => 'API_SECRET' }) driver = Selenium::WebDriver.for(:remote, url: 'https://hub.testingbot.com/wd/hub', options: options ) ``` To see how to do this, please select a combination of browser, version and platform in the drop-down menus below. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a Ruby WebDriver test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```ruby #!/usr/bin/env ruby require 'selenium-webdriver' options = Selenium::WebDriver::Chrome::Options.new options.add_option('platformName', 'WIN11') options.add_option('browserVersion', 'latest') options.add_option('tb:options', { 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'Ruby Tunnel Test' }) # Point to the tunnel running on localhost:4445 driver = Selenium::WebDriver.for( :remote, url: 'http://localhost:4445/wd/hub', options: options ) driver.navigate.to 'https://www.google.com/ncr' element = driver.find_element(:name, 'q') element.send_keys 'TestingBot', :return puts driver.title driver.quit ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. To get started, please see the examples on [parallel\_tests](https://github.com/grosser/parallel_tests). ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. You can use our [Ruby API client](https://github.com/testingbot/testingbot_ruby) to report back test results. Install it with `gem install testingbot`: ```ruby require 'testingbot' api = TestingBot::Api.new('API_KEY', 'API_SECRET') api.update_test(driver.session_id, { name: 'My Test', success: true, build: 'build-123' }) ``` ## Other Ruby Framework examples - [Capybara](https://testingbot.com/support/web-automate/selenium/ruby/capybara) Capybara is an integration testing tool for rack based web applications. - [Cucumber](https://testingbot.com/support/web-automate/selenium/ruby/cucumber) Cucumber is a Ruby based test tool for BDD. - [RSpec](https://testingbot.com/support/web-automate/selenium/ruby/rspec) RSpec is a behavior-driven development (BDD) framework, inspired by JBehave. - [Test::Unit](https://testingbot.com/support/web-automate/selenium/ruby/testunit) Test-Unit is a xUnit family unit testing framework for Ruby. - [Minitest](https://testingbot.com/support/web-automate/selenium/ruby/minitest) Minitest is a minimal replacement for test-unit with a complete suite of testing facilities. - [Watir](https://testingbot.com/support/web-automate/selenium/ruby/watir) Watir, pronounced water, is an open-source family of Ruby libraries for automating web browsers. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/ruby/capybara # Capybara Automated Testing See our [Cucumber + Capybara example repository](https://github.com/testingbot/ruby-cucumber-capybara-example). TestingBot supports Selenium testing using Capybara. Before we start with the example, please install Capybara and the TestingBot client gem. The example below drives Capybara with Cucumber and runs in parallel via `parallel_tests`, so install those too: ```bash gem install capybara testingbot cucumber parallel_tests ``` Now let's start with a simple Capybara test case. ```yaml # Google Feature Feature: Google Search Functionality Background: Given I am on https://www.google.com/ncr Scenario: Can find search results When I fill in "q" found by "name" with "TestingBot" And I submit Then I should see title "TestingBot - Google Search" ``` ```ruby # Google Steps Given /^I am on (.*)$/ do |url| visit url end When /^I fill in "([^\"]*)" found by "([^\"]*)" with "([^\"]*)"$/ do |value, type, keys| fill_in(value, :with => keys) end When /^I submit$/ do find_field('q').native.send_key(:enter) end Then /^I should see title "([^\"]*)"$/ do |title| expect(page).to have_title title end ``` ## Integrate with TestingBot Please use this module to run Capybara tests on TestingBot: ```ruby require "capybara/cucumber" require "selenium/webdriver" require "testingbot" Capybara.default_max_wait_time = 10 Capybara.default_driver = :testingbot Before do |scenario| jobname = scenario.name Capybara.register_driver :testingbot do |app| options = Selenium::WebDriver::Options.chrome options.browser_version = ENV['version'] options.platform_name = ENV['platform'] options.add_option('tb:options', { name: jobname }) url = "https://#{ENV['TB_KEY']}:#{ENV['TB_SECRET']}@hub.testingbot.com/wd/hub" Capybara::Selenium::Driver.new(app, browser: :remote, url: url, options: options ) end Capybara.session_name = "#{jobname} - #{ENV['platform']} - #{ENV['browserName']} - #{ENV['version']}" @driver = Capybara.current_session.driver @session_id = @driver.browser.session_id puts "TestingBotSessionId=#{@session_id} job-name=#{jobname}" end After do |scenario| @driver.quit api = TestingBot::Api.new(ENV['TB_KEY'], ENV['TB_SECRET']) api.update_test(@session_id, success: !scenario.exception) end ``` This module uses a Rake file which contains the browsers we want to run the test on: **Rakefile** : ```ruby def run_tests(platform, browser, version, junit_dir) system("platform=\"#{platform}\" browserName=\"#{browser}\" version=\"#{version}\" JUNIT_DIR=\"#{junit_dir}\" parallel_cucumber features -n 20") end task :win11_chrome_latest do run_tests('WIN11', 'chrome', 'latest', 'junit_reports/win11_chrome_latest') end task :sonoma_chrome_latest do run_tests('SONOMA', 'chrome', 'latest', 'junit_reports/sonoma_chrome_latest') end multitask :test_testingbot => [ :win11_chrome_latest, :sonoma_chrome_latest, ] do puts 'Running automation' end ``` To start the test, please run this command: ```bash bundle exec rake test_testingbot ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field, inside the `env.rb` file. ```ruby options = Selenium::WebDriver::Options.chrome options.browser_version = ENV['version'] options.platform_name = ENV['platform'] options.add_option('tb:options', { 'name' => 'My Capybara Test' }) ``` To see how to do this, please select a combination of browser, version and platform in the drop-down menus below. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a Capybara test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: **env.rb** : ```ruby Capybara.register_driver :testingbot do |app| options = Selenium::WebDriver::Options.chrome options.browser_version = 'latest' options.platform_name = 'WIN11' options.add_option('tb:options', { 'name' => 'Tunnel Test' }) Capybara::Selenium::Driver.new(app, browser: :remote, url: 'http://localhost:4445/wd/hub', options: options ) end Capybara.default_driver = :testingbot ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. To start running tests in parallel, add more browser configurations in the `Rakefile`: **Rakefile** : ```ruby def run_tests(platform, browser, version, junit_dir) system("platform=\"#{platform}\" browserName=\"#{browser}\" version=\"#{version}\" JUNIT_DIR=\"#{junit_dir}\" parallel_cucumber features -n 20") end task :win11_chrome_latest do run_tests('WIN11', 'chrome', 'latest', 'junit_reports/win11_chrome_latest') end task :sonoma_chrome_latest do run_tests('SONOMA', 'chrome', 'latest', 'junit_reports/sonoma_chrome_latest') end multitask :test_testingbot => [ :win11_chrome_latest, :sonoma_chrome_latest, ] do puts 'Running automation' end ``` The `-n 20` in the example above specifies how many parallel sessions to run. Usually you would use the same number as the maximum parallel sessions allowed by your TestingBot plan. To run your tests in parallel, run this command: ```bash bundle exec rake test_testingbot ``` ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. You can use our [Ruby API client](https://github.com/testingbot/testingbot_ruby) to report back test results. ```ruby api = TestingBot::Api.new("API_KEY", "API_SECRET") api.update_test(Capybara.current_session.driver.browser.session_id, name: 'My Test', success: true) ``` ## Other Ruby Framework examples - [Capybara](https://testingbot.com/support/web-automate/selenium/ruby/capybara) Capybara is an integration testing tool for rack based web applications. - [Cucumber](https://testingbot.com/support/web-automate/selenium/ruby/cucumber) Cucumber is a Ruby based test tool for BDD. - [RSpec](https://testingbot.com/support/web-automate/selenium/ruby/rspec) RSpec is a behavior-driven development (BDD) framework, inspired by JBehave. - [Test::Unit](https://testingbot.com/support/web-automate/selenium/ruby/testunit) Test-Unit is a xUnit family unit testing framework for Ruby. - [Minitest](https://testingbot.com/support/web-automate/selenium/ruby/minitest) Minimal (mostly drop-in) replacement for test-unit. - [Watir](https://testingbot.com/support/web-automate/selenium/ruby/watir) Watir, pronounced water, is an open-source (BSD) family of Ruby libraries for automating web browsers. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/ruby/cucumber # Cucumber Automated Testing See our [Cucumber example repository](https://github.com/testingbot/ruby-cucumber-example). To get started, please make sure you have installed the necessary gems: ```ruby source 'https://rubygems.org' gem 'cucumber' gem 'selenium-webdriver' gem 'rspec' gem 'rake' gem 'parallel' gem 'testingbot' ``` Save the block above as a `Gemfile` and run `bundle install`. Cucumber expects a specific project layout: feature files in `features/`, step definitions in `features/step_definitions/`, and the driver setup/hooks in `features/support/env.rb`. ## Example Next, we'll create a `features` and a `steps` file. Combined, they're your test. ### Example feature (features/google.feature) ```yaml Feature: Google can search Background: Given I am on Google Scenario: Search for a term When I fill in "q" found by "name" with "TestingBot" And I submit Then I should see title "TestingBot - Google Search" ``` ### Example steps (features/step\_definitions/steps.rb) ```ruby Given /^I am on (.+)$/ do |url| @browser.navigate.to "https://www.google.com" end When /^I fill in "([^"]*)" found by "([^"]*)" with "([^"]*)"$/ do |value, type, keys| @element = @browser.find_element(type, value) @element.send_keys keys end When /^I submit$/ do @element.submit end Then /^I should see title "([^"]*)"$/ do |title| raise "Fail" if @browser.title != title end ``` ### Example env.rb (features/support/env.rb) ```ruby require 'selenium-webdriver' require 'testingbot' Before do options = Selenium::WebDriver::Options.send(ENV['browserName'] || 'chrome') options.browser_version = ENV['version'] || 'latest' options.platform_name = ENV['platform'] || 'WIN11' options.add_option('tb:options', { key: ENV['TB_KEY'], secret: ENV['TB_SECRET'] }) url = 'https://hub.testingbot.com/wd/hub' client = Selenium::WebDriver::Remote::Http::Default.new client.open_timeout = 180 @browser = Selenium::WebDriver.for(:remote, url: url, options: options, http_client: client) end After do |test_case| sessionid = @browser.session_id jobname = test_case.name puts "TestingBotSessionId=#{sessionid} job-name=#{jobname}" @browser.quit end ``` Now we can run the test on TestingBot, to start, please run this command: ```bash cucumber TB_KEY=api_key TB_SECRET=api_secret platform=WIN11 browserName=chrome ``` ### Configuring capabilities To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before:** ```ruby driver = Selenium::WebDriver.for :chrome ``` **After:** ```ruby driver = Selenium::WebDriver.for(:remote, url: 'https://hub.testingbot.com/wd/hub', options: options) ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ```ruby options = Selenium::WebDriver::Options.chrome options.browser_version = 'latest' options.platform_name = 'WIN11' options.add_option('tb:options', { key: 'api_key', secret: 'api_secret' }) driver = Selenium::WebDriver.for(:remote, url: 'https://hub.testingbot.com/wd/hub', options: options) ``` To see how to do this, please select a combination of browser, version and platform in the drop-down menus below. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a Cucumber test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ### Example env.rb ```ruby require 'selenium-webdriver' options = Selenium::WebDriver::Options.send(ENV['SELENIUM_BROWSER'] || 'chrome') options.browser_version = ENV['SELENIUM_VERSION'] || 'latest' options.platform_name = ENV['SELENIUM_PLATFORM'] || 'WIN11' options.add_option('tb:options', { name: 'My first Test', key: ENV['TB_KEY'], secret: ENV['TB_SECRET'] }) url = 'http://localhost:4445/wd/hub' browser = Selenium::WebDriver.for(:remote, url: url, options: options) Before do |scenario| @browser = browser end at_exit do browser.quit end ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. To start running tests in parallel, first we'll define a `browsers.json` file which contains all the browser combinations: ```json [{ "browserName": "firefox", "browserVersion": "latest", "platformName": "SONOMA" }, { "browserName": "chrome", "browserVersion": "latest", "platformName": "WIN11" }] ``` Now, we'll create a single test ( **features** and **steps** file) which we'll use to run on the browser combinations we defined above. ```yaml Feature: Google can search Background: Given I am on Google Scenario: Search for a term When I fill in "q" found by "name" with "TestingBot" And I submit Then I should see title "TestingBot - Google Search" ``` ```ruby Given /^I am on (.+)$/ do |url| @browser.navigate.to "https://www.google.com" end When /^I fill in "([^"]*)" found by "([^"]*)" with "([^"]*)"$/ do |value, type, keys| @element = @browser.find_element(type, value) @element.send_keys keys end When /^I submit$/ do @element.submit end Then /^I should see title "([^"]*)"$/ do |title| raise "Fail" if @browser.title != title end ``` Now we need to create a Ruby script that will run the test above, and use the environment variables to determine browser/OS combination. ```ruby require 'selenium-webdriver' options = Selenium::WebDriver::Options.send(ENV['SELENIUM_BROWSER']) options.browser_version = ENV['SELENIUM_VERSION'] options.platform_name = ENV['SELENIUM_PLATFORM'] options.add_option('tb:options', { key: ENV['TB_KEY'], secret: ENV['TB_SECRET'], build: ENV['TB_AUTOMATE_BUILD'] }) url = 'https://hub.testingbot.com/wd/hub' browser = Selenium::WebDriver.for(:remote, url: url, options: options) Before do |scenario| @browser = browser end at_exit do browser.quit end ``` Finally, we create a `Rakefile` which handles the running of the tests in parallel. ```ruby require 'rubygems' require 'cucumber' require 'cucumber/rake/task' require 'parallel' require 'json' @browsers = JSON.load(open('browsers.json')) @parallel_limit = ENV["nodes"] || 1 @parallel_limit = @parallel_limit.to_i task :cucumber do Parallel.each(@browsers, :in_processes => @parallel_limit) do |browser| begin puts "Running with: #{browser.inspect}" ENV['SELENIUM_BROWSER'] = browser['browserName'] ENV['SELENIUM_VERSION'] = browser['browserVersion'] ENV['SELENIUM_PLATFORM'] = browser['platformName'] Rake::Task[:run_features].execute() rescue StandardError => e puts "Error while running task: #{e.message}" end end end Cucumber::Rake::Task.new(:run_features) task :default => [:cucumber] ``` To run your tests in parallel, run this command: ```bash rake TB_KEY=api_key TB_SECRET=api_secret nodes=3 ``` ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. You can use our [Ruby API client](https://github.com/testingbot/testingbot_ruby) to report back test results. Call this from an `After` hook in `features/support/env.rb`, where the driver is available as `@browser` (and read the result via `test_case.failed?`): ```ruby After do |test_case| api = TestingBot::Api.new("key", "secret") api.update_test(@browser.session_id, { name: test_case.name, success: !test_case.failed? }) @browser.quit end ``` ## Other Ruby Framework examples - [Capybara](https://testingbot.com/support/web-automate/selenium/ruby/capybara) Capybara is an integration testing tool for rack based web applications. - [Cucumber](https://testingbot.com/support/web-automate/selenium/ruby/cucumber) Cucumber is a Ruby based test tool for BDD. - [RSpec](https://testingbot.com/support/web-automate/selenium/ruby/rspec) RSpec is a behavior-driven development (BDD) framework, inspired by JBehave. - [Test::Unit](https://testingbot.com/support/web-automate/selenium/ruby/testunit) Test-Unit is a xUnit family unit testing framework for Ruby. - [Minitest](https://testingbot.com/support/web-automate/selenium/ruby/minitest) Minimal (mostly drop-in) replacement for test-unit. - [Watir](https://testingbot.com/support/web-automate/selenium/ruby/watir) Watir, pronounced water, is an open-source (BSD) family of Ruby libraries for automating web browsers. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/ruby/rspec # RSpec Automated Testing See our [RSpec example repository](https://github.com/testingbot/ruby-rspec-example) for a simple example on how to run RSpec tests in parallel on TestingBot. First, install the required gems: ```bash gem install rspec selenium-webdriver testingbot ``` Next, let's create a `testingbot_driver.rb` file which makes it easy to run tests on TestingBot: ```ruby require "selenium/webdriver" module TestingbotDriver class << self def testingbot_endpoint "https://hub.testingbot.com/wd/hub" end def new_driver options = Selenium::WebDriver::Chrome::Options.new options.add_option('platformName', 'WIN11') options.add_option('browserVersion', 'latest') options.add_option('tb:options', { 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'RSpec Test' }) Selenium::WebDriver.for :remote, url: testingbot_endpoint, options: options end end end ``` Now, we need to tell RSpec to use our custom driver to run the test. The best way to do this, is by defining an around hook: **spec\_helper.rb** : ```ruby require "rspec" require_relative "testingbot_driver" RSpec.configure do |config| config.around(:example) do |example| @driver = TestingbotDriver.new_driver begin example.run ensure @driver.quit end end end ``` Now, let's create an actual test that will use TestingBot: ```bash rspec example.rb ``` ```ruby require_relative "spec_helper" describe "Google's Search Functionality" do it "can find search results" do @driver.manage.timeouts.implicit_wait = 10 @driver.navigate.to "https://www.google.com" raise "Unable to load Google." unless @driver.title.include? "Google" query = @driver.find_element :name, "q" query.send_keys "TestingBot" query.submit puts @driver.title end end ``` ## Configuring capabilities To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before:** ```ruby driver = Selenium::WebDriver.for :chrome ``` **After:** ```ruby options = Selenium::WebDriver::Chrome::Options.new options.add_option('platformName', 'WIN11') options.add_option('browserVersion', 'latest') options.add_option('tb:options', { 'key' => 'API_KEY', 'secret' => 'API_SECRET' }) driver = Selenium::WebDriver.for(:remote, url: "https://hub.testingbot.com/wd/hub", options: options ) ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ```ruby options = Selenium::WebDriver::Chrome::Options.new options.add_option('platformName', 'WIN11') options.add_option('browserVersion', 'latest') options.add_option('tb:options', { 'key' => 'API_KEY', 'secret' => 'API_SECRET' }) driver = Selenium::WebDriver.for(:remote, url: "https://hub.testingbot.com/wd/hub", options: options ) ``` To see how to do this, please select a combination of browser, version and platform in the drop-down menus below. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a RSpec test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: **testingbot\_driver.rb** ```ruby require "selenium/webdriver" module TestingbotDriver class << self def testingbot_endpoint "http://localhost:4445/wd/hub" end def new_driver options = Selenium::WebDriver::Chrome::Options.new options.add_option('platformName', 'WIN11') options.add_option('browserVersion', 'latest') options.add_option('tb:options', { 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'RSpec Tunnel Test' }) Selenium::WebDriver.for :remote, url: testingbot_endpoint, options: options end end end ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. Add the parallel\_tests gem to your Gemfile: ```ruby gem "rspec" gem "parallel_tests" ``` Now create some more tests in the same spec directory. Make sure the tests also end with `_spec.rb` in the filename. Once you have some more tests, you can run the tests in parallel with this command: ```bash bundle exec parallel_rspec -n 2 spec/ ``` The `-n 2` means how many tests you will run concurrently. This should match up with the number of parallel tests you have with your TestingBot paid plan or 2 for the free trial. ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. You can use our [Ruby API client](https://github.com/testingbot/testingbot_ruby) to report back test results. Wire it into the `around` hook in `spec_helper.rb` so the status reflects the actual result. Read the session id and outcome **before** calling `@driver.quit` (the session id is no longer available after quit): ```ruby require 'testingbot' RSpec.configure do |config| config.around(:example) do |example| @driver = TestingbotDriver.new_driver begin example.run ensure api = TestingBot::Api.new('API_KEY', 'API_SECRET') api.update_test(@driver.session_id, { name: example.description, success: example.exception.nil? }) @driver.quit end end end ``` ## Other Ruby Framework examples - [Capybara](https://testingbot.com/support/web-automate/selenium/ruby/capybara) Capybara is an integration testing tool for rack based web applications. - [Cucumber](https://testingbot.com/support/web-automate/selenium/ruby/cucumber) Cucumber is a Ruby based test tool for BDD. - [RSpec](https://testingbot.com/support/web-automate/selenium/ruby/rspec) RSpec is a behavior-driven development (BDD) framework, inspired by JBehave. - [Test::Unit](https://testingbot.com/support/web-automate/selenium/ruby/testunit) Test-Unit is a xUnit family unit testing framework for Ruby. - [Minitest](https://testingbot.com/support/web-automate/selenium/ruby/minitest) Minitest is a minimal replacement for test-unit with a complete suite of testing facilities. - [Watir](https://testingbot.com/support/web-automate/selenium/ruby/watir) Watir, pronounced water, is an open-source family of Ruby libraries for automating web browsers. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/ruby/testunit # Test::Unit Automated Testing See our [TestUnit example repository](https://github.com/testingbot/ruby-testunit-example) for a simple example on how to run TestUnit tests in parallel on TestingBot. ## Installation Install the required gems. Note that `test/unit` was removed from Ruby's standard library, so the `test-unit` gem is required on modern Ruby versions: ```bash gem install selenium-webdriver test-unit testingbot ``` ## Example Test::Unit ```ruby require 'selenium-webdriver' require 'test/unit' class SampleTest < Test::Unit::TestCase def setup options = Selenium::WebDriver::Chrome::Options.new options.add_option('platformName', 'WIN11') options.add_option('browserVersion', 'latest') options.add_option('tb:options', { 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'Test::Unit Example' }) @driver = Selenium::WebDriver.for( :remote, url: 'https://hub.testingbot.com/wd/hub', options: options ) end def test_google_search @driver.navigate.to 'https://www.google.com/ncr' element = @driver.find_element(:name, 'q') element.send_keys 'TestingBot' element.submit assert_equal('TestingBot - Google Search', @driver.title) end def teardown @driver.quit end end ``` ## Configuring capabilities To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before:** ```ruby driver = Selenium::WebDriver.for :chrome ``` **After:** ```ruby options = Selenium::WebDriver::Chrome::Options.new options.add_option('platformName', 'WIN11') options.add_option('browserVersion', 'latest') options.add_option('tb:options', { 'key' => 'API_KEY', 'secret' => 'API_SECRET' }) driver = Selenium::WebDriver.for(:remote, url: 'https://hub.testingbot.com/wd/hub', options: options) ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ```ruby options = Selenium::WebDriver::Chrome::Options.new options.add_option('platformName', 'WIN11') options.add_option('browserVersion', 'latest') options.add_option('tb:options', { 'key' => 'API_KEY', 'secret' => 'API_SECRET' }) driver = Selenium::WebDriver.for(:remote, url: 'https://hub.testingbot.com/wd/hub', options: options) ``` To see how to do this, please select a combination of browser, version and platform in the drop-down menus below. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a Ruby WebDriver test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```ruby require 'selenium-webdriver' require 'test/unit' class SampleTest < Test::Unit::TestCase def setup options = Selenium::WebDriver::Chrome::Options.new options.add_option('platformName', 'WIN11') options.add_option('browserVersion', 'latest') options.add_option('tb:options', { 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'Tunnel Test' }) # Point to the tunnel running on localhost:4445 @driver = Selenium::WebDriver.for( :remote, url: 'http://localhost:4445/wd/hub', options: options ) end def test_internal_site @driver.navigate.to 'http://localhost:3000' # Your internal website element = @driver.find_element(:css, 'h1') assert(element.text.include?('Welcome')) end def teardown @driver.quit end end ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. The example below demonstrates how you can easily run tests simultaneously in a variety of browsers. First, you need to add the [parallel testing gem](https://github.com/grosser/parallel_tests) to your Gemfile. This Gem will allow you to run multiple tests simultaneously. Create 2 test files: `test_chrome.rb` and `test_firefox.rb` **test\_chrome.rb** ```ruby require 'selenium-webdriver' require 'test/unit' class ChromeTest < Test::Unit::TestCase def setup options = Selenium::WebDriver::Chrome::Options.new options.add_option('platformName', 'WIN11') options.add_option('browserVersion', 'latest') options.add_option('tb:options', { 'key' => ENV['TB_KEY'], 'secret' => ENV['TB_SECRET'], 'name' => 'Parallel Chrome Test' }) @driver = Selenium::WebDriver.for( :remote, url: 'https://hub.testingbot.com/wd/hub', options: options ) end def test_google_search @driver.navigate.to 'https://www.google.com' element = @driver.find_element(:name, 'q') element.send_keys 'TestingBot' element.submit assert_equal('TestingBot - Google Search', @driver.title) end def teardown @driver.quit end end ``` **test\_firefox.rb** ```ruby require 'selenium-webdriver' require 'test/unit' class FirefoxTest < Test::Unit::TestCase def setup options = Selenium::WebDriver::Firefox::Options.new options.add_option('platformName', 'WIN11') options.add_option('browserVersion', 'latest') options.add_option('tb:options', { 'key' => ENV['TB_KEY'], 'secret' => ENV['TB_SECRET'], 'name' => 'Parallel Firefox Test' }) @driver = Selenium::WebDriver.for( :remote, url: 'https://hub.testingbot.com/wd/hub', options: options ) end def test_google_search @driver.navigate.to 'https://www.google.com' element = @driver.find_element(:name, 'q') element.send_keys 'TestingBot' element.submit assert_equal('TestingBot - Google Search', @driver.title) end def teardown @driver.quit end end ``` Now you need to instruct the parallel\_test gem to run these two tests in parallel: ```bash TB_KEY="API_KEY" TB_SECRET="API_SECRET" parallel_test -n 2 test_chrome.rb test_firefox.rb ``` ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. You can use our [Ruby API client](https://github.com/testingbot/testingbot_ruby) to report back test results. ```ruby api = TestingBot::Api.new('API_KEY', 'API_SECRET') api.update_test(@driver.session_id, { name: 'My Test', success: true }) ``` ## Other Ruby Framework examples - [Capybara](https://testingbot.com/support/web-automate/selenium/ruby/capybara) Capybara is an integration testing tool for rack based web applications. - [Cucumber](https://testingbot.com/support/web-automate/selenium/ruby/cucumber) Cucumber is a Ruby based test tool for BDD. - [RSpec](https://testingbot.com/support/web-automate/selenium/ruby/rspec) RSpec is a behavior-driven development (BDD) framework, inspired by JBehave. - [Minitest](https://testingbot.com/support/web-automate/selenium/ruby/minitest) Minimal (mostly drop-in) replacement for test-unit. - [Watir](https://testingbot.com/support/web-automate/selenium/ruby/watir) Watir, pronounced water, is an open-source (BSD) family of Ruby libraries for automating web browsers. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/ruby/minitest # Minitest Automated Testing Minitest is a very fast unit testing framework which supports TDD, BDD, mocking and benchmarking. To begin, please make sure you have Minitest and the other required gems installed on your system: ```bash gem install minitest selenium-webdriver testingbot ``` ## Example code ```ruby require 'minitest/autorun' require 'selenium-webdriver' class GoogleTest < Minitest::Test def setup options = Selenium::WebDriver::Chrome::Options.new options.browser_version = 'latest' options.platform_name = 'WIN11' options.add_option('tb:options', { 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'Minitest Example' }) @driver = Selenium::WebDriver.for(:remote, url: 'https://hub.testingbot.com/wd/hub', options: options ) end def test_google_search @driver.navigate.to 'https://www.google.com' element = @driver.find_element(:name, 'q') element.send_keys 'TestingBot' element.submit assert_includes @driver.title, 'TestingBot' end def teardown @driver.quit end end ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ```ruby driver = Selenium::WebDriver.for(:remote, url: 'https://hub.testingbot.com/wd/hub', options: options ) ``` To see how to do this, please select a combination of browser, version and platform in the drop-down menus below. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a Ruby WebDriver test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```ruby require 'minitest/autorun' require 'selenium-webdriver' class GoogleTest < Minitest::Test def setup options = Selenium::WebDriver::Chrome::Options.new options.browser_version = 'latest' options.platform_name = 'WIN11' options.add_option('tb:options', { 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'Minitest Tunnel Example' }) @driver = Selenium::WebDriver.for(:remote, url: 'http://localhost:4445/wd/hub', options: options ) end def test_google_search @driver.navigate.to 'https://www.google.com' element = @driver.find_element(:name, 'q') element.send_keys 'TestingBot' element.submit assert_includes @driver.title, 'TestingBot' end def teardown @driver.quit end end ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. The example below demonstrates how you can easily run tests simultaneously in a variety of browsers. Create a `browsers.json` file with the [list of browsers](https://testingbot.com/support/web-automate/browsers) you want to use to run tests in parallel: ```json [ { "browserName": "firefox", "browserVersion": "latest", "platformName": "WIN11" }, { "browserName": "safari", "browserVersion": "18", "platformName": "SEQUOIA" }, { "browserName": "chrome", "browserVersion": "latest", "platformName": "WIN11" } ] ``` Now we need to create the test and use environment variables which we will pass to the test script. ```ruby require 'minitest/autorun' require 'selenium-webdriver' class GoogleTest < Minitest::Test def setup # Dynamically select browser options based on environment options_class = case ENV['TB_BROWSERNAME'] when 'firefox' then Selenium::WebDriver::Firefox::Options when 'safari' then Selenium::WebDriver::Safari::Options else Selenium::WebDriver::Chrome::Options end options = options_class.new options.browser_version = ENV['TB_VERSION'] options.platform_name = ENV['TB_PLATFORM'] options.add_option('tb:options', { 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'Parallel Test' }) @driver = Selenium::WebDriver.for(:remote, url: 'https://hub.testingbot.com/wd/hub', options: options ) end def test_google_search @driver.navigate.to 'https://www.google.com' element = @driver.find_element(:name, 'q') element.send_keys 'TestingBot' element.submit assert_includes @driver.title, 'TestingBot' end def teardown @driver.quit end end ``` Now we need to create a `Rakefile` which will start the tests in parallel. The Rakefile uses the `rake` and `parallel` gems, so install them first: ```bash gem install rake parallel ``` ```ruby require 'rake/testtask' require 'parallel' require 'json' @browsers = JSON.load(File.open('browsers.json')) @test_folder = 'test/*_test.rb' @parallel_limit = (ENV['nodes'] || 1).to_i task :minitest do current_browser = '' begin Parallel.map(@browsers, in_threads: @parallel_limit) do |browser| current_browser = browser puts "Running Browser: #{browser.inspect}" # Pass env vars per child process so parallel threads don't race on the shared ENV hash. child_env = { 'TB_BROWSERNAME' => browser['browserName'], 'TB_VERSION' => browser['browserVersion'], 'TB_PLATFORM' => browser['platformName'] } Dir.glob(@test_folder).each do |test_file| IO.popen(child_env, "ruby #{test_file}") do |io| io.each { |line| puts line } end end end rescue SystemExit, Interrupt puts 'User stopped script!' puts "Failed to run tests for #{current_browser.inspect}" end end task default: [:minitest] ``` ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. You can use our [Ruby API client](https://github.com/testingbot/testingbot_ruby) to report back test results. ```ruby api = TestingBot::Api.new("API_KEY", "API_SECRET") api.update_test(@driver.session_id, name: 'My Test', success: true) ``` ## Other Ruby Framework examples - [Capybara](https://testingbot.com/support/web-automate/selenium/ruby/capybara) Capybara is an integration testing tool for rack based web applications. - [Cucumber](https://testingbot.com/support/web-automate/selenium/ruby/cucumber) Cucumber is a Ruby based test tool for BDD. - [RSpec](https://testingbot.com/support/web-automate/selenium/ruby/rspec) RSpec is a behavior-driven development (BDD) framework, inspired by JBehave. - [Test::Unit](https://testingbot.com/support/web-automate/selenium/ruby/testunit) Test-Unit is a xUnit family unit testing framework for Ruby. - [Watir](https://testingbot.com/support/web-automate/selenium/ruby/watir) Watir, pronounced water, is an open-source (BSD) family of Ruby libraries for automating web browsers. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/ruby/watir # Watir WebDriver Automated Testing See our [Watir example repository](https://github.com/testingbot/ruby-rspec-watir-example) for a simple example on how to run Watir tests in parallel on TestingBot. Watir, pronounced water, is an open-source (BSD) family of Ruby libraries for automating web browsers. To install the Watir gem: ```bash gem install watir ``` ## Example with Watir ```ruby require 'watir' require 'selenium-webdriver' options = Selenium::WebDriver::Chrome::Options.new options.add_option('platformName', 'WIN11') options.add_option('browserVersion', 'latest') options.add_option('tb:options', { 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'Watir Test' }) browser = Watir::Browser.new( :remote, url: 'https://hub.testingbot.com/wd/hub', options: options ) browser.goto 'https://www.google.com/ncr' browser.text_field(name: 'q').set 'TestingBot' browser.text_field(name: 'q').send_keys :enter puts browser.title browser.quit ``` ## Configuring capabilities To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before:** ```ruby browser = Watir::Browser.new(:firefox) ``` **After:** ```ruby options = Selenium::WebDriver::Firefox::Options.new options.add_option('platformName', 'WIN11') options.add_option('browserVersion', 'latest') options.add_option('tb:options', { 'key' => 'API_KEY', 'secret' => 'API_SECRET' }) browser = Watir::Browser.new( :remote, url: 'https://hub.testingbot.com/wd/hub', options: options ) ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ```ruby options = Selenium::WebDriver::Chrome::Options.new options.add_option('platformName', 'WIN11') options.add_option('browserVersion', 'latest') options.add_option('tb:options', { 'key' => 'API_KEY', 'secret' => 'API_SECRET' }) browser = Watir::Browser.new( :remote, url: 'https://hub.testingbot.com/wd/hub', options: options ) ``` To see how to do this, please select a combination of browser, version and platform in the drop-down menus below. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a Watir test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar API_KEY API_SECRET ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```ruby require 'watir' require 'selenium-webdriver' options = Selenium::WebDriver::Firefox::Options.new options.add_option('platformName', 'WIN11') options.add_option('browserVersion', 'latest') options.add_option('tb:options', { 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'Watir Tunnel Test' }) browser = Watir::Browser.new( :remote, url: 'http://localhost:4445/wd/hub', options: options ) browser.goto 'https://www.google.com/ncr' browser.text_field(name: 'q').set 'TestingBot' browser.text_field(name: 'q').send_keys :enter puts browser.title browser.quit ``` ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. You can use our [Ruby API client](https://github.com/testingbot/testingbot_ruby) to report back test results. Install it with `gem install testingbot`: ```ruby require 'testingbot' api = TestingBot::Api.new('API_KEY', 'API_SECRET') api.update_test(browser.driver.session_id, { name: 'My Test', success: true }) ``` ## Other Ruby Framework examples - [Capybara](https://testingbot.com/support/web-automate/selenium/ruby/capybara) Capybara is an integration testing tool for rack based web applications. - [Cucumber](https://testingbot.com/support/web-automate/selenium/ruby/cucumber) Cucumber is a Ruby based test tool for BDD. - [RSpec](https://testingbot.com/support/web-automate/selenium/ruby/rspec) RSpec is a behavior-driven development (BDD) framework, inspired by JBehave. - [Test::Unit](https://testingbot.com/support/web-automate/selenium/ruby/testunit) Test-Unit is a xUnit family unit testing framework for Ruby. - [Minitest](https://testingbot.com/support/web-automate/selenium/ruby/minitest) Minimal (mostly drop-in) replacement for test-unit. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/robotframework # Robot Framework Automated WebDriver Testing See our [Robot Framework example repository](https://github.com/testingbot/robotframework-web-example) for a first example on how to run Robot Framework tests on TestingBot. With [SeleniumLibrary](https://github.com/robotframework/SeleniumLibrary) you can run WebDriver tests with [Robot Framework](https://robotframework.org/). Robot Framework is a test automation framework to run acceptance tests. The Robot test framework allows you to build easy to read test cases, using keyword driven and behavior-driven approaches. The framework provides reports, in HTML format. ## Setting up Robot Framework Make sure you've installed [pip](https://pip.pypa.io/en/stable/), then install the necessary libraries: ```bash pip install robotframework-seleniumlibrary requests ``` ### Running your first test Below is an example on how to run a simple test on Chrome. When the test has finished, the test name and success state is sent to TestingBot so you can see the test success/failures in our dashboard. Save the example below in a `test.robot` file: [Browser Options](https://testingbot.com#) [Desired Capabilities (deprecated)](https://testingbot.com#) This tab uses `desired_capabilities=`, which was removed in SeleniumLibrary 6.0 (April 2023). Use the **Browser Options** tab for modern SeleniumLibrary. ```robotframework ***Settings*** Library SeleniumLibrary Library TestingBot Test Setup Open test browser Test Teardown Close test browser ***Variables*** ${CREDENTIALS} key:secret ***Test Cases*** Simple Test Go to https://www.google.com Page should contain Google ***Keywords*** Open test browser Open browser about: chrome ... remote_url=https://${CREDENTIALS}@hub.testingbot.com/wd/hub ... desired_capabilities=browserName:${BROWSER},browserVersion:${VERSION},platformName:${PLATFORM} Close test browser Report TestingBot status ... ${SUITE_NAME} | ${TEST_NAME} ... ${TEST_STATUS} ${CREDENTIALS} Close all browsers ``` Save the code below in a `TestingBot.py` file: ```python import requests from robot.libraries.BuiltIn import BuiltIn def report_testingbot_status(name, status, credentials): selenium = BuiltIn().get_library_instance('SeleniumLibrary') session_id = selenium.driver.session_id payload = {'test[name]': name, 'test[success]': int(status == 'PASS')} key, secret = credentials.split(':') url = f'https://api.testingbot.com/v1/tests/{session_id}' response = requests.put(url, data=payload, auth=(key, secret)) assert response.status_code == 200, response.text ``` To run the test, run this command: ```bash PYTHONPATH=$PYTHONPATH:. robot --variable BROWSER:chrome --variable VERSION:latest --variable PLATFORM:WIN11 test.robot ``` Once the test finished running, you should see it in the [member dashboard](https://testingbot.com/members). Use the variables to run the test on different platforms/browsers/versions. ```robotframework ***Settings*** Library SeleniumLibrary Library TestingBot Library BrowserOptions Test Setup Open test browser Test Teardown Close test browser ***Test Cases*** Simple Test Go to https://www.google.com Page should contain Google ***Keywords*** Open test browser ${options}= Get Browser Options ${BROWSER} # Set capabilities Call Method ${options} set_capability platformName ${PLATFORM} Call Method ${options} set_capability browserName ${BROWSER} Call Method ${options} set_capability browserVersion ${VERSION} # Set custom TestingBot capabilities using tb:options ${tb_options}= Create Dictionary key=${TB_KEY} secret=${TB_SECRET} name=test build=MyTestBuild Call Method ${options} set_capability tb:options ${tb_options} # Open browser with options Open Browser remote_url=https://hub.testingbot.com/wd/hub options=${options} Close test browser Report TestingBot status ... ${SUITE_NAME} | ${TEST_NAME} ... ${TEST_STATUS} ${TB_KEY}:${TB_SECRET} Close all browsers ``` Save the code below in a `TestingBot.py` file: ```python import requests from robot.libraries.BuiltIn import BuiltIn def report_testingbot_status(name, status, credentials): selenium = BuiltIn().get_library_instance('SeleniumLibrary') session_id = selenium.driver.session_id payload = {'test[name]': name, 'test[success]': int(status == 'PASS')} key, secret = credentials.split(':') url = f'https://api.testingbot.com/v1/tests/{session_id}' response = requests.put(url, data=payload, auth=(key, secret)) assert response.status_code == 200, response.text ``` Now we'll need to specify the multiple browser options in a file called `BrowserOptions.py`: ```python from selenium.webdriver import ChromeOptions, FirefoxOptions, EdgeOptions, SafariOptions def get_browser_options(browser): if browser.lower() == 'chrome': return ChromeOptions() elif browser.lower() == 'firefox': return FirefoxOptions() elif browser.lower() == 'edge': return EdgeOptions() elif browser.lower() == 'safari': return SafariOptions() else: raise ValueError(f"Unsupported browser: {browser}") ``` To run the test, run this command: ```bash PYTHONPATH=$PYTHONPATH:. robot --variable BROWSER:chrome --variable VERSION:latest --variable PLATFORM:WIN11 --variable TB_KEY:API_KEY --variable TB_SECRET:API_SECRET test.robot ``` Once the test finished running, you should see it in the [member dashboard](https://testingbot.com/members). Use the variables to run the test on different platforms/browsers/versions. ### Specifying the operating system, browser and version To let TestingBot know on which browser/platform you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ```robotframework Call Method ${options} set_capability browserName ${BROWSER} Call Method ${options} set_capability browserVersion ${VERSION} Call Method ${options} set_capability platformName ${PLATFORM} ``` To see how to do this, please select a combination of browser, version and platform in the drop-down menus below. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a Robot Framework test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back. ## Parallel Testing You can run multiple Robot Framework tests in parallel by using the [Pabot](https://pabot.org/) library. First install Pabot using pip: ```bash pip install -U robotframework-pabot ``` Then run your tests using the `pabot` command instead of `robot`: ```bash PYTHONPATH=. pabot --processes 2 --loglevel TRACE --variable BROWSER:chrome --variable VERSION:latest --variable PLATFORM:WIN11 --variable TB_KEY:API_KEY --variable TB_SECRET:API_SECRET test.robot ``` Some recommendations when running tests in parallel: - Make sure each test case is independent and does not rely on shared state. - Specify the `--processes` with the same amount as your TestingBot plan allows (parallel test sessions). - Consider using `--processtimeout` to set a timeout for each process. ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. You can use our [Python API client](https://github.com/testingbot/testingbotclient) to report back test results. ```python from testingbotclient import TestingBotClient client = TestingBotClient('API_KEY', 'API_SECRET') client.tests.update_test( driver.session_id, name='My Test', passed=True, build='build-123' ) ``` ## Other Python Framework examples - [PyTest](https://testingbot.com/support/web-automate/selenium/python/pytest) PyTest makes it easy to run Selenium tests with Python. - [Behave](https://testingbot.com/support/web-automate/selenium/python/behave) Behave is behaviour-driven development, Python style. - [Lettuce](https://testingbot.com/support/web-automate/selenium/python/lettuce) Lettuce is a Python BDD plugin based on Ruby's Cucumber, offering Gherkin stories. - [PyUnit](https://testingbot.com/support/web-automate/selenium/python/pyunit) PyUnit is the standard unit testing framework module for Python, described as a Python version of JUnit. - [Helium](https://testingbot.com/support/web-automate/selenium/python/helium) Helium is a tool that makes it easy to test websites and automate browsers. - [Pylenium](https://testingbot.com/support/web-automate/selenium/python/pylenium) Pylenium brings the best of Selenium, Cypress and Python into one package. - [SeleniumBase](https://testingbot.com/support/web-automate/selenium/python/seleniumbase) SeleniumBase is a Python package that simplifies Selenium testing. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/nodejs ### NodeJS Examples: - [WebdriverIO](https://testingbot.com/support/web-automate/selenium/nodejs/webdriverio) - [Nightwatch](https://testingbot.com/support/web-automate/selenium/nodejs/nightwatch) - [TestCafe](https://testingbot.com/support/web-automate/selenium/nodejs/testcafe) - [CodeceptJS](https://testingbot.com/support/web-automate/selenium/nodejs/codeceptjs) - [Hermione](https://testingbot.com/support/web-automate/selenium/nodejs/hermione) - [Jest](https://testingbot.com/support/web-automate/selenium/nodejs/jest) # NodeJS Automated Testing First, make sure you have installed NodeJS. If you haven't, you can easily install it by downloading NodeJS from [the NodeJS website](https://nodejs.org/). ## Getting started With TestingBot you can easily run your automated tests with [any NodeJS test framework](https://testingbot.com#frameworks), here's a simple example with **selenium-webdriver** : ```bash npm install selenium-webdriver ``` ```javascript const webdriver = require('selenium-webdriver'); const TB_KEY = process.env.TB_KEY || 'your_key'; const TB_SECRET = process.env.TB_SECRET || 'your_secret'; const capabilities = { 'platformName': 'WIN11', 'browserName': 'chrome', 'browserVersion': 'latest', 'tb:options': { 'name': 'NodeJS Sample Test' } }; async function runTest() { const driver = await new webdriver.Builder() .usingServer(`https://${TB_KEY}:${TB_SECRET}@hub.testingbot.com/wd/hub`) .withCapabilities(capabilities) .build(); await driver.get("https://testingbot.com"); await driver.wait(webdriver.until.titleMatches(/TestingBot/i), 5000); await driver.quit(); } runTest(); ``` This test will start a Chrome browser on Windows in our cloud, go to TestingBot, and verify the page title contains "TestingBot". Your test results are now available on the commandline interface, and in our member area. Make sure to always stop your test (`driver.quit()`), otherwise it will continue running, leading to a timeout. ### Configuring capabilities To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before:** ```javascript const driver = new webdriver.Builder() .usingServer('http://localhost:4444/wd/hub') .withCapabilities({ 'browserName': 'firefox' }).build(); ``` **After:** ```javascript const TB_KEY = process.env.TB_KEY || 'your_key'; const TB_SECRET = process.env.TB_SECRET || 'your_secret'; const driver = new webdriver.Builder() .usingServer(`https://${TB_KEY}:${TB_SECRET}@hub.testingbot.com/wd/hub`) .withCapabilities({ 'browserName': 'firefox', 'browserVersion': 'latest', 'platformName': 'WIN11', 'tb:options': { 'name': 'My Test' } }).build(); ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. Looking to generate more custom capabilities? Use the [Selenium capabilities generator](https://testingbot.com/support/web-automate/selenium/capabilities). ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a NodeJS test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```javascript const webdriver = require('selenium-webdriver'); const TB_KEY = process.env.TB_KEY || 'your_key'; const TB_SECRET = process.env.TB_SECRET || 'your_secret'; const capabilities = { 'platformName': 'WIN11', 'browserName': 'chrome', 'browserVersion': 'latest', 'tb:options': { 'name': 'NodeJS Sample Test' } }; async function runTest() { const driver = await new webdriver.Builder() .usingServer(`http://${TB_KEY}:${TB_SECRET}@localhost:4445/wd/hub`) .withCapabilities(capabilities) .build(); await driver.get("https://testingbot.com"); await driver.wait(webdriver.until.titleMatches(/TestingBot/i), 5000); await driver.quit(); } runTest(); ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. To run tests in parallel, we recommend using a test runner like [WebDriverIO](https://testingbot.com/support/web-automate/selenium/nodejs/webdriverio) to run your tests simultaneously. ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. Install our NPM package to interact with the TestingBot API: ```bash npm install testingbot-api ``` After your test completed, you can send back the test success state and other meta-data: ```javascript const TestingBot = require('testingbot-api'); const tb = new TestingBot({ api_key: "api_key", api_secret: "api_secret" }); // 'sessionId' is the WebDriver session id of your test (see "Get Session ID" below) tb.updateTest({ 'test[name]': 'Example Test', 'test[success]': true }, sessionId, (err, testDetails) => { if (err) { console.error(err); } }); ``` To retrieve the `sessionId` of the test, please see [Get Session ID](https://testingbot.com/support/web-automate/selenium/get-session-id?type=nodejs). ## Other NodeJS Framework examples - [WebDriverIO](https://testingbot.com/support/web-automate/selenium/nodejs/webdriverio) With WebDriverIO you can run Mocha, Jasmine and Cucumber tests. - [CodeceptJS](https://testingbot.com/support/web-automate/selenium/nodejs/codeceptjs) Run acceptance tests with CodeceptJS on TestingBot. - [Nightwatch](https://testingbot.com/support/web-automate/selenium/nodejs/nightwatch) Nightwatch is an automated testing framework written in NodeJS. - [Testplane (Hermione)](https://testingbot.com/support/web-automate/selenium/nodejs/hermione) Testplane (formerly Hermione) is a test framework similar to WebDriverIO, with automatic test retries and plugins. - [Jest](https://testingbot.com/support/web-automate/selenium/nodejs/jest) Use the Jest test runner with selenium-webdriver to run end-to-end browser tests. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/nodejs/nightwatch # Nightwatch Automated Browser Testing See the TestingBot [NightWatch example repository](https://github.com/testingbot/nightwatch-example) for an example on how to run NightWatch tests in parallel on TestingBot. Install Nightwatch: ```bash npm install nightwatch testingbot-api --save-dev ``` Now, you'll need to create a settings file so that Nightwatch knows it needs to connect to the TestingBot Selenium Grid. Create a `nightwatch.conf.js` file with the following contents. ```javascript const testingBotOptions = { 'tb:options' : { "build" : "nightwatch-example", "name" : "Nightwatch Example Test", "seleniumVersion" : "4.24.0", key: process.env.TESTINGBOT_KEY || 'YOUR_KEY', secret: process.env.TESTINGBOT_SECRET || 'YOUR_SECRET', }, } const testingBot = { webdriver: { start_process: false, host: 'hub.testingbot.com', port: 443, default_path_prefix: '/wd/hub', ssl: true, timeout_options: { timeout: 120000, retry_attempts: 3 }, keep_alive: true }, desiredCapabilities: { browserName: 'chrome', ...testingBotOptions } } const nightwatchConfigs = { src_folders: ['tests'], custom_commands_path: 'custom_commands', output_folder: 'reports', live_output: true, test_settings: { default: { launch_url: 'https://testingbot.com' }, "testingbot.chrome": { ...testingBot, desiredCapabilities:{ browserName: 'chrome', ...testingBotOptions } }, "testingbot.firefox": { ...testingBot, desiredCapabilities:{ browserName: 'firefox', ...testingBotOptions } }, "testingbot.edge": { ...testingBot, desiredCapabilities:{ browserName: 'edge', ...testingBotOptions } } } } module.exports = nightwatchConfigs; ``` We will also need to add a test script, let's create a new directory called `tests` and add a file: `googleTest.js` ```javascript const https = require('https'); module.exports = { '@tags': ['test'], 'Google': function(client) { client .url('https://testingbot.com') .waitForElementVisible('body', 1000) .assert.textContains('body', 'TestingBot') .end(); }, afterEach: function(client, done) { client.customTestingBotEnd(); done(); } }; ``` To report back the status to TestingBot, we'll need to create a custom command in the `custom_commands` directory. Please create a file called `customTestingBotEnd.js` in this directory. ```javascript exports.command = function(callback) { const TestingBot = require("testingbot-api"); const tb = new TestingBot({ api_key: process.env.TESTINGBOT_KEY, api_secret: process.env.TESTINGBOT_SECRET }); const sessionid = this.sessionId, jobName = this.currentTest.name, passed = this.currentTest.results.testcases[jobName].failed === 0; console.log("TestingBotSessionId=" + sessionid); tb.updateTest({ 'test[success]': passed, 'test[name]': jobName }, sessionid, () => { this.end(callback) }); }; ``` To run the test, please run this command: ```bash TESTINGBOT_KEY=... TESTINGBOT_SECRET=... ./node_modules/.bin/nightwatch --test ./tests/googleTest.js --env testingbot.chrome ``` ## Configuring capabilities To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before:** ```javascript "webdriver" : { "start_process" : false, "host" : "localhost", "port" : 4444 } ``` **After:** ```javascript "webdriver" : { "start_process" : false, "host" : "hub.testingbot.com", "port" : 443, "default_path_prefix" : "/wd/hub", "ssl" : true } ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. Looking to generate more custom capabilities? Use the [Selenium capabilities generator](https://testingbot.com/support/web-automate/selenium/capabilities). ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a NodeJS test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```json { "src_folders" : ["tests/"], "webdriver" : { "start_process" : false, "host" : "localhost", "port" : 4445, "default_path_prefix" : "/wd/hub" }, "test_settings" : { "default" : { "desiredCapabilities": { "browserName": "firefox", "platformName": "WIN11", "browserVersion": "latest" } } } } ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. To run tests in parallel, you can use the following command to specify multiple capabilities at once. ```bash ./node_modules/.bin/nightwatch --test ./tests/googleTest.js --env testingbot.chrome,testingbot.edge,testingbot.firefox ``` ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. Install our NPM package to interact with the TestingBot API: ```bash npm install testingbot-api ``` After your test completed, you can send back the test success state and other meta-data: ```javascript exports.command = function(callback) { const TestingBot = require("testingbot-api"); const tb = new TestingBot({ api_key: process.env.TESTINGBOT_KEY, api_secret: process.env.TESTINGBOT_SECRET }); const sessionid = this.sessionId, jobName = this.currentTest.name, passed = this.currentTest.results.testcases[jobName].failed === 0; console.log("TestingBotSessionId=" + sessionid); tb.updateTest({ 'test[success]': passed, 'test[name]': jobName }, sessionid, () => { this.end(callback) }); }; ``` ## Other NodeJS Framework examples - [WebDriverIO](https://testingbot.com/support/web-automate/selenium/nodejs/webdriverio) With WebDriverIO you can run Mocha, Jasmine and Cucumber tests. - [CodeceptJS](https://testingbot.com/support/web-automate/selenium/nodejs/codeceptjs) Run acceptance tests with CodeceptJS on TestingBot. - [Protractor](https://testingbot.com/support/web-automate/selenium/nodejs/protractor) Protractor is an end-to-end test framework for AngularJS applications. Protractor is a NodeJS program built on top of WebDriverJS. - [Nightwatch](https://testingbot.com/support/web-automate/selenium/nodejs/nightwatch) Nightwatch is an automated testing framework written in NodeJS. - [Hermione](https://testingbot.com/support/web-automate/selenium/nodejs/hermione) Hermione is a Test Framework similar to WebDriverIO, with automatic test retries and plugins. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/nodejs/protractor Protractor is deprecated and no longer actively maintained by the Angular team. Consider using [WebDriverIO](https://testingbot.com/support/web-automate/selenium/nodejs/webdriverio) or [Playwright](https://testingbot.com/support/web-automate/playwright) for new projects. # Protractor Automated Browser Testing Install Protractor: ```bash npm install -g protractor ``` Now, you'll need to create a config file so that Protractor knows it needs to connect to our Selenium Grid. Create a file called `conf.js`: ```javascript exports.config = { seleniumAddress: 'https://hub.testingbot.com/wd/hub', specs: ['spec.js'], maxSessions: 6, // 6 TOTAL sessions across all capabilities multiCapabilities: [{ // in 1 chrome run the 10 specs sequentially browserName: 'chrome', platformName: 'WIN11', browserVersion: 'latest', 'tb:options': { key: "API_KEY", secret: "API_SECRET", name: 'My First Protractor test' } }, { // 5 chrome sessions will start, each running 1 spec file at a time, until all 10 finish browserName: 'chrome', platformName: 'SEQUOIA', browserVersion: 'latest-1', shardTestFiles: true, maxInstances: 5, 'tb:options': { key: "API_KEY", secret: "API_SECRET", name: 'My First Protractor test' } }, { // Same as repeating the previous setup twice browserName: 'chrome', platformName: 'WIN11', browserVersion: 'latest-2', shardTestFiles: true, maxInstances: 5, count: 2, 'tb:options': { key: "API_KEY", secret: "API_SECRET", name: 'My First Protractor test' } }] } ``` Next, we'll create a `spec.js` (which we defined above in the conf file). This will contain the test logic. ```javascript describe('Protractor Demo App', function() { it('should have a title', function() { browser.get('https://juliemr.github.io/protractor-demo/'); expect(browser.getTitle()).toEqual('Super Calculator'); }); }); ``` To run the test, please run this command: ```bash protractor conf.js ``` ## Configuring capabilities To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before:** ```javascript exports.config = { multiCapabilities: [{ browserName: 'chrome' }] } ``` **After:** ```javascript exports.config = { seleniumAddress: 'https://hub.testingbot.com/wd/hub', multiCapabilities: [{ browserName: 'chrome', platformName: 'WIN11', browserVersion: 'latest', 'tb:options': { key: "API_KEY", secret: "API_SECRET" } }] } ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a Protractor test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```javascript exports.config = { seleniumAddress: 'http://localhost:4445/wd/hub', specs: ['spec.js'], multiCapabilities: [{ browserName: 'chrome', platformName: 'WIN11', browserVersion: 'latest', 'tb:options': { key: "API_KEY", secret: "API_SECRET" } }] } ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. To run tests in parallel, please see the following example: ```javascript exports.config = { capabilities: { browserName: 'chrome', count: 2 // number of parallel tests } } ``` ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. ```javascript exports.config = { seleniumAddress: 'https://hub.testingbot.com/wd/hub', specs: ['spec.js'], maxSessions: 6, // 6 TOTAL sessions across all capabilities multiCapabilities: [{ // in 1 chrome run the 10 specs sequentially browserName: 'chrome', platformName: 'WIN11', browserVersion: 'latest', 'tb:options': { key: "API_KEY", secret: "API_SECRET", name: 'My First Protractor test' } }, { // 5 chrome sessions will start, each running 1 spec file at a time, until all 10 finish browserName: 'chrome', platformName: 'SEQUOIA', browserVersion: 'latest-1', shardTestFiles: true, maxInstances: 5, 'tb:options': { key: "API_KEY", secret: "API_SECRET", name: 'My First Protractor test' } }, { // Same as repeating the previous setup twice browserName: 'chrome', platformName: 'WIN11', browserVersion: 'latest-2', shardTestFiles: true, maxInstances: 5, count: 2, 'tb:options': { key: "API_KEY", secret: "API_SECRET", name: 'My First Protractor test' } }], onComplete: function (passed) { const TestingBot = require('testingbot-api'); const tb = new TestingBot({ api_key: process.env.TESTINGBOT_KEY, api_secret: process.env.TESTINGBOT_SECRET }); return browser.getSession().then(function (session) { return new Promise(function (resolve, reject) { tb.updateTest({ 'test[success]': passed }, session.getId(), function (err) { err ? reject(err) : resolve(); }); }); }); } } ``` Please see the example below, where we use the `onComplete` callback provided by Protractor together with the [testingbot-api](https://github.com/testingbot/testingbot-api) client. ```javascript onComplete: function (passed) { const TestingBot = require('testingbot-api'); const tb = new TestingBot({ api_key: process.env.TESTINGBOT_KEY, api_secret: process.env.TESTINGBOT_SECRET }); return browser.getSession().then(function (session) { return new Promise(function (resolve, reject) { tb.updateTest({ 'test[success]': passed }, session.getId(), function (err) { err ? reject(err) : resolve(); }); }); }); } ``` This `onComplete` handler will be invoked after every Protractor test. The test status will be sent to TestingBot, where we will display the pass/fail state in your test dashboard. ## Other NodeJS Framework examples - [WebDriverIO](https://testingbot.com/support/web-automate/selenium/nodejs/webdriverio) With WebDriverIO you can run Mocha, Jasmine and Cucumber tests. - [CodeceptJS](https://testingbot.com/support/web-automate/selenium/nodejs/codeceptjs) Run acceptance tests with CodeceptJS on TestingBot. - [Protractor](https://testingbot.com/support/web-automate/selenium/nodejs/protractor) Protractor is an end-to-end test framework for AngularJS applications. Protractor is a NodeJS program built on top of WebDriverJS. - [Nightwatch](https://testingbot.com/support/web-automate/selenium/nodejs/nightwatch) Nightwatch is an automated testing framework written in NodeJS. - [WD.js](https://testingbot.com/support/web-automate/selenium/nodejs/wd) WD.js is a NodeJS client for WebDriver/Selenium. - [Hermione](https://testingbot.com/support/web-automate/selenium/nodejs/hermione) Hermione is a Test Framework similar to WebDriverIO, with automatic test retries and plugins. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/nodejs/hermione # Hermione Testing **Note:** Hermione has been renamed to [Testplane](https://github.com/gemini-testing/testplane). The examples below still work, but you may want to migrate to Testplane for future development. Hermione is a test framework inspired upon WebDriverIO. If you are already familiar with WebDriverIO and Mocha, you will enjoy working with the Hermione Test Framework. There are several advantages that Hermione has over WebDriverIO: - **Runs tests in subprocesses** : to take advantage of all your CPU cores. - **Extensible** : You can add custom commands in the hermione config and use them as `browser.myCustomCommand` in tests. - **Built-in assert library** : Hermione provides a global `expect` variable that gives you access to a number of matchers that let you validate different things on the browser, an element or mock object. - **Retry failed tests** : To prevent flaky tests, hermione retries a failed test before marking it as failed. ## Setting up Hermione To set up Hermione, you can enter the following command in your terminal. First, make sure you've created a directory where you will set up a new Hermione project. ```bash npm init hermione-app . ``` You will be asked a couple of questions, which will help Hermione to decide how to configure your configuration file (`.hermione.conf.js`). Once the configuration file has been created, you'll need to edit the browsers block and add your TestingBot key and secret. Here's an example of what it should look like: ```javascript module.exports = { gridUrl: 'https://hub.testingbot.com/wd/hub', baseUrl: 'http://localhost', pageLoadTimeout: 0, httpTimeout: 60000, testTimeout: 90000, resetCursor: false, sets: { desktop: { files: [ 'tests/**/*.hermione.js' ], browsers: [ 'chrome' ] } }, browsers: { chrome: { automationProtocol: 'webdriver', desiredCapabilities: { browserName: 'chrome', browserVersion: 'latest', platformName: 'WIN11', 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, name: 'My Hermione Test' } } } }, plugins: { 'html-reporter/hermione': { // https://github.com/gemini-testing/html-reporter enabled: true, path: 'hermione-report', defaultView: 'all', diffMode: '3-up-scaled' } } } ``` The `html-reporter` plugin used above is a separate package that is not installed by `npm init hermione-app`. Install it before running your tests: ```bash npm install html-reporter --save-dev ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a Hermione test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```javascript module.exports = { gridUrl: 'http://localhost:4445/wd/hub', baseUrl: 'http://localhost', ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. By default, Hermione will run all tests simultaneously. We advise to set the `parallelLimit` option to the same number of parallel tests as your TestingBot plan allows. This will prevent tests from queueing up and eventually timing out. ```javascript module.exports = { parallelLimit: 2 // depends on the TestingBot plan you have } ``` ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. Use the [testingbot-api](https://github.com/testingbot/testingbot-api) client to send the test status back from a Hermione hook: ```javascript const TestingBot = require('testingbot-api'); const tb = new TestingBot({ api_key: process.env.TESTINGBOT_KEY, api_secret: process.env.TESTINGBOT_SECRET }); // In hermione.conf.js, register an afterEach hook afterEach(async function() { const passed = this.currentTest.state === 'passed'; await new Promise((resolve, reject) => { tb.updateTest({ 'test[success]': passed, 'test[name]': this.currentTest.title }, browser.sessionId, (err) => err ? reject(err) : resolve()); }); }); ``` ## Other NodeJS Framework examples - [WebDriverIO](https://testingbot.com/support/web-automate/selenium/nodejs/webdriverio) With WebDriverIO you can run Mocha, Jasmine and Cucumber tests. - [CodeceptJS](https://testingbot.com/support/web-automate/selenium/nodejs/codeceptjs) Run acceptance tests with CodeceptJS on TestingBot. - [Protractor](https://testingbot.com/support/web-automate/selenium/nodejs/protractor) Protractor is an end-to-end test framework for AngularJS applications. - [Soda](https://testingbot.com/support/web-automate/selenium/nodejs/soda) Selenium Node Adapter. A light-weight Selenium RC client for NodeJS. - [Nightwatch](https://testingbot.com/support/web-automate/selenium/nodejs/nightwatch) Nightwatch is an automated testing framework written in NodeJS. - [WD.js](https://testingbot.com/support/web-automate/selenium/nodejs/wd) WD.js is a NodeJS client for WebDriver/Selenium. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/nodejs/codeceptjs # Getting started with CodeceptJS [CodeceptJS](https://codecept.io/) is an end-to-end testing framework which supports Selenium, Appium, Playwright, Puppeteer and more. It offers scenario driven testing, PageObjects support, beautiful reports and much more. Let's start with installing everything we need: ```bash npm i --save-dev codeceptjs@3 webdriverio @codeceptjs/configure @wdio/testingbot-service@7 ``` The TestingBot reporting/tunnel integration below uses CodeceptJS's built-in `wdio` plugin together with the `@wdio/testingbot-service` package. This requires **CodeceptJS 3** (the bundled `wdio` plugin was removed in CodeceptJS 4) and **`@wdio/testingbot-service` 7** (version 8+ is ESM-only and cannot be loaded by the plugin). The `webdriverio` package is required by the `WebDriver` helper. Now we need to set up CodeceptJS. Please use the command below to initialize a new project: ```bash npx codeceptjs init ``` This will ask a bunch of questions, and will create a sample test at the end of the process. ## Example test Now that you have everything set up, you can create a very simple test: ```javascript Feature('My First Test'); Scenario('test something', ({ I }) => { I.amOnPage('https://testingbot.com'); I.see('TestingBot'); }); ``` This simple test will navigate to TestingBot and verify if the name TestingBot is present on the page. Before we can run this test, we need to configure CodeceptJS to connect and run a browser on our grid. ## Configuration You can configure CodeceptJS with a `codecept.conf.js` file: ```javascript const { setHeadlessWhen } = require('@codeceptjs/configure'); // turn on headless mode when running with HEADLESS=true environment variable // HEADLESS=true npx codecept run setHeadlessWhen(process.env.HEADLESS); exports.config = { tests: './*_test.js', output: './output', helpers: { WebDriver: { // 'url' is the base URL of the application under test (passed to I.amOnPage('/')). url: 'https://www.google.com', host: 'hub.testingbot.com', port: 443, protocol: 'https', path: '/wd/hub', browser: 'chrome', capabilities: { browserName: 'chrome', browserVersion: 'latest', platformName: 'WIN11', 'tb:options': { build: 'CodeceptJS', key: 'YOUR KEY HERE', secret: 'YOUR SECRET HERE' } }, smartWait: 5000, restart: false, timeouts: { script: 60000, 'page load': 10000 } } }, include: { I: './steps_file.js' }, bootstrap: null, mocha: {}, name: 'codeceptjs-example', plugins: { retryFailedStep: { enabled: true }, screenshotOnFail: { enabled: true }, wdio: { enabled: true, services: ['testingbot'], user: 'YOUR KEY HERE', key: 'YOUR SECRET HERE' } } } ``` To run the test, you can do: ```bash npx codeceptjs run --steps ``` This will start the latest Chrome browser on TestingBot and run the test on our grid. You will see a test appear in your member dashboard on TestingBot, together with a video, log files and other assets generated by the test. ## Specify Browsers & Devices To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before** ```javascript // codecept.conf.js exports.config = { tests: './*_test.js', output: './output', helpers: { WebDriver: {} } }; ``` **After** ```javascript // codecept.conf.js plugins: { ... wdio: { enabled: true, services: ['testingbot'], user: 'YOUR KEY HERE', key: 'YOUR SECRET HERE' } } ``` ### Specifying the operating system, browser and version To let TestingBot know on which browser/platform you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. To see how to do this, please select a combination of browser, version and platform in the drop-down menus below. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Reporting Test Results If you are using the [wdio-testingbot-service](https://webdriver.io/docs/testingbot-service/) then your tests will automatically report back meta-data to TestingBot (like test success, name, stacktrace, ...) ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a CodeceptJS test with our Tunnel: Adjust your `codecept.conf.js` configuration to use `tbTunnel: true`. The [wdio-testingbot-service](https://webdriver.io/docs/testingbot-service/) will automatically download and use the tunnel for your tests. **codecept.conf.js** ```javascript plugins: { ... wdio: { enabled: true, services: ['testingbot'], user: 'YOUR KEY HERE', key: 'YOUR SECRET HERE', tbTunnel: true } } ``` ## Run tests in Parallel Please see [CodeceptJS parallel testing](https://codecept.io/parallel/#parallel-execution-by-workers) on how to run your tests in parallel. You can specify the concurrency by specifying a number of workers: ```bash npx codeceptjs run-workers 2 ``` or run multiple different browsers with: ```bash npx codeceptjs run-multiple --all ``` ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Other Options We offer many other [test options](https://testingbot.com/support/web-automate/selenium/test-options), for example: disable video recording, specifying a custom Firefox Profile, loading Chrome/Firefox/Safari extensions, running an executable before your test starts, uploading files, ... See our [list of test options](https://testingbot.com/support/web-automate/selenium/test-options) for a full list of options to customize your tests. ## Other NodeJS Framework examples - [WebDriverIO](https://testingbot.com/support/web-automate/selenium/nodejs/webdriverio) With WebDriverIO you can run Mocha, Jasmine and Cucumber tests. - [CodeceptJS](https://testingbot.com/support/web-automate/selenium/nodejs/codeceptjs) Run acceptance tests with CodeceptJS on TestingBot. - [Protractor](https://testingbot.com/support/web-automate/selenium/nodejs/protractor) Protractor is an end-to-end test framework for AngularJS applications. Protractor is a NodeJS program built on top of WebDriverJS. - [Soda](https://testingbot.com/support/web-automate/selenium/nodejs/soda) Selenium Node Adapter. A light-weight Selenium RC client for NodeJS. - [Nightwatch](https://testingbot.com/support/web-automate/selenium/nodejs/nightwatch) Nightwatch is an automated testing framework written in NodeJS. - [WD.js](https://testingbot.com/support/web-automate/selenium/nodejs/wd) WD.js is a NodeJS client for WebDriver/Selenium. - [Hermione](https://testingbot.com/support/web-automate/selenium/nodejs/hermione) Hermione is a Test Framework similar to WebDriverIO, with automatic test retries and plugins. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/nodejs/soda # NodeJS with Soda **Note:** Soda is a Selenium RC (Selenium 1.0) client that has been deprecated. We recommend using modern alternatives like [WebDriverIO](https://testingbot.com/support/web-automate/selenium/nodejs/webdriverio), [Nightwatch](https://testingbot.com/support/web-automate/selenium/nodejs/nightwatch), or [Playwright](https://testingbot.com/support/web-automate/playwright) for new projects. Install the TestingBot plugin: ```bash npm install testingbot ``` You can now run your Selenium tests with NodeJS on our grid. Run this example: ```bash node example.js ``` The example below uses the legacy Selenium 1 (RC) protocol via the `testingbot` npm package. TestingBot's hub no longer accepts the RC protocol, so this code will not run as-is. It is kept here for historical reference only, for working Node.js code, see the "After" snippet below or [our Node.js guide](https://testingbot.com/support/web-automate/selenium/nodejs). ```javascript var soda = require('testingbot'), assert = require('assert'); var browser = soda.createTestingBotClient({ 'url': 'https://www.google.com', 'os': 'Windows', 'browser': 'chrome', 'browser-version': 'latest', 'max-duration': 300 }); // Log commands as they are fired browser.on('command', function(cmd, args) { console.log(' \x1b[33m%s\x1b[0m: %s', cmd, args.join(', ')); }); browser .chain .session() .setTimeout(8000) .open('/') .waitForPageToLoad(5000) .getTitle(function(title) { assert.ok(title.includes('Google')) }) .testComplete() .end(function(err) { if (err) { console.log(err); } }); ``` ## Configuring capabilities To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before:** ```javascript const driver = new webdriver.Builder() .usingServer('http://localhost:4444/wd/hub') .withCapabilities({ browserName: 'firefox' }) .build(); ``` **After:** ```javascript const driver = new webdriver.Builder() .usingServer('https://hub.testingbot.com/wd/hub') .withCapabilities({ browserName: 'firefox', browserVersion: 'latest', platformName: 'WIN11', 'tb:options': { key: 'API_KEY', secret: 'API_SECRET' } }) .build(); ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a NodeJS test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```javascript const { Builder } = require('selenium-webdriver'); const capabilities = { browserName: 'chrome', browserVersion: 'latest', platformName: 'WIN11', 'tb:options': { key: 'API_KEY', secret: 'API_SECRET', name: 'Tunnel Test' } }; // Point to the tunnel running on localhost:4445 const driver = new Builder() .usingServer('http://localhost:4445/wd/hub') .withCapabilities(capabilities) .build(); (async () => { await driver.get('https://www.google.com'); const title = await driver.getTitle(); console.log(title); await driver.quit(); })(); ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. To run tests in parallel, we recommend using a test runner like [WebDriverIO](https://testingbot.com/support/web-automate/selenium/nodejs/webdriverio) to run your tests simultaneously. ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. Install our NPM package to interact with the TestingBot API: ```bash npm install testingbot-api ``` After your test completed, you can send back the test success state and other meta-data: ```javascript const TestingBot = require('testingbot-api'); const tb = new TestingBot({ api_key: 'API_KEY', api_secret: 'API_SECRET' }); // Obtain the session id from your WebDriver session after the test runs: // const session = await driver.getSession(); // const sessionId = session.getId(); tb.updateTest({ 'test[name]': 'My Test', 'test[success]': true, 'test[build]': 'build-123' }, sessionId, (error, response) => { if (error) console.error(error); }); ``` ## Other NodeJS Framework examples - [WebDriverIO](https://testingbot.com/support/web-automate/selenium/nodejs/webdriverio) With WebDriverIO you can run Mocha, Jasmine and Cucumber tests. - [CodeceptJS](https://testingbot.com/support/web-automate/selenium/nodejs/codeceptjs) Run acceptance tests with CodeceptJS on TestingBot. - [Protractor](https://testingbot.com/support/web-automate/selenium/nodejs/protractor) Protractor is an end-to-end test framework for AngularJS applications. **Note:** Protractor is deprecated and no longer maintained. - [Nightwatch](https://testingbot.com/support/web-automate/selenium/nodejs/nightwatch) Nightwatch is an automated testing framework written in NodeJS. - [WD.js](https://testingbot.com/support/web-automate/selenium/nodejs/wd) WD.js is a NodeJS client for WebDriver/Selenium. - [Hermione](https://testingbot.com/support/web-automate/selenium/nodejs/hermione) Hermione is a Test Framework similar to WebDriverIO, with automatic test retries and plugins. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/nodejs/wd # NodeJS Automated Testing with WD.js WD.js is no longer actively maintained. We recommend using [WebDriverIO](https://testingbot.com/support/web-automate/selenium/nodejs/webdriverio) for new projects. WD.js is a NodeJS client for WebDriver/Selenium. Let's start with making sure WD.js is installed: ```bash npm install wd ``` You are now ready to run a WebDriver test with NodeJS on our grid. Please take a look at the following example: ```javascript const wd = require('wd'); const chai = require('chai'); const chaiAsPromised = require('chai-as-promised'); chai.use(chaiAsPromised); chai.should(); const capabilities = { 'browserName': 'chrome', 'platformName': 'WIN11', 'browserVersion': 'latest', 'tb:options': { 'key': 'API_KEY', 'secret': 'API_SECRET', 'name': 'WD.js Test' } }; describe("using promises and chai-as-promised", function() { let browser; before(function() { browser = wd.promiseChainRemote('hub.testingbot.com', 443, '', '', 'https'); return browser.init(capabilities); }); beforeEach(function() { return browser.get("https://testingbot.com/"); }); after(function() { return browser.quit(); }); it("should retrieve the page title", function() { return browser.title().should.eventually.include("TestingBot"); }); }); ``` ## Configuring capabilities To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this: **Before** ```javascript browser = wd.promiseChainRemote(); browser.init({ browserName: 'chrome' }) ``` **After** ```javascript const capabilities = { 'browserName': 'chrome', 'platformName': 'WIN11', 'browserVersion': 'latest', 'tb:options': { 'key': 'API_KEY', 'secret': 'API_SECRET' } }; browser = wd.promiseChainRemote('hub.testingbot.com', 443, '', '', 'https'); browser.init(capabilities); ``` ### Specifying the operating system, browser and version To let TestingBot know on which browser/platform you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. To see how to do this, please select a combination of browser, version and platform in the drop-down menus below. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Other Options We offer many other [test options](https://testingbot.com/support/web-automate/selenium/test-options), for example: disable video recording, specifying a custom Firefox Profile, loading Chrome/Firefox/Safari extensions, running an executable before your test starts, uploading files, ... See our [list of test options](https://testingbot.com/support/web-automate/selenium/test-options) for a full list of options to customize your tests. ## Other NodeJS Framework examples - [WebDriverIO (Recommended)](https://testingbot.com/support/web-automate/selenium/nodejs/webdriverio) With WebDriverIO you can run Mocha, Jasmine and Cucumber tests. - [CodeceptJS](https://testingbot.com/support/web-automate/selenium/nodejs/codeceptjs) Run acceptance tests with CodeceptJS on TestingBot. - [Nightwatch](https://testingbot.com/support/web-automate/selenium/nodejs/nightwatch) Nightwatch is an automated testing framework written in NodeJS. - [TestCafe](https://testingbot.com/support/web-automate/selenium/nodejs/testcafe) TestCafe is a NodeJS framework for end-to-end web testing. - [Hermione](https://testingbot.com/support/web-automate/selenium/nodejs/hermione) Hermione is a Test Framework similar to WebDriverIO, with automatic test retries and plugins. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/nodejs/webdriverio ### WebDriverIO Examples: - [Cucumber](https://testingbot.com#cucumber) - [Mocha](https://testingbot.com#mocha) - [Jasmine](https://testingbot.com#jasmine) # NodeJS Automated Testing with WebdriverIO With WebDriverIO you can run [Mocha](https://testingbot.com#mocha), [Jasmine](https://testingbot.com#jasmine) (v2.0) and [Cucumber](https://testingbot.com#cucumber) tests. See our [WebdriverIO examples repository](https://github.com/testingbot/webdriverio-examples) for some examples on how to use WebdriverIO with TestingBot. TestingBot has its own [WebDriverIO service](https://webdriver.io/docs/testingbot-service.html) plugin which makes configuring your tests easier and which sends test meta-data back to TestingBot. Let's start with making sure `webdriverio` is installed: ```bash npm i --save-dev @wdio/cli @wdio/local-runner ``` ## WebDriverIO TestingBot Service WebDriverIO contains a [TestingBot service](https://webdriver.io/docs/testingbot-service.html) which makes it easy to run tests with WebDriver.io on TestingBot. With this service, WebDriverIO will start a [TestingBot Tunnel](https://testingbot.com/support/tunnel) if required, and send back test meta-data (passed state, name, build-id, ...) In order to use the service you need to install it: ```bash npm install @wdio/testingbot-service --save-dev ``` ## Cucumber Example To run Cucumber tests on TestingBot, please follow these steps: ```bash npm install @wdio/testingbot-service --save-dev npm install @wdio/dot-reporter --save-dev npm install @wdio/cucumber-framework --save-dev npm install @cucumber/cucumber@^10 --save-dev ``` The step definitions below import from `@cucumber/cucumber`, so install it explicitly and match the major version bundled by `@wdio/cucumber-framework` (v10) to avoid loading two different Cucumber instances. Now we'll make a simple Cucumber test which uses Firefox on TestingBot to go to Google.com and verify the page's title. Please create the following files: **wdio.conf.js** ```javascript exports.config = { /** * specify test files */ specs: [ './features/*.feature' ], /** * capabilities */ capabilities: [{ browserName: 'chrome', browserVersion: 'latest', platformName: 'WIN11' }], /** * test configurations */ logLevel: 'silent', waitforTimeout: 10000, connectionRetryTimeout: 480000, framework: 'cucumber', services: [ ['testingbot'] ], user: 'API_KEY', key: 'API_SECRET', reporters: ['dot'], cucumberOpts: { require: ['./step-definitions.js'] } }; ``` **step-definitions.js** ```javascript const { Given, When, Then } = require('@cucumber/cucumber') const assert = require('assert') Given('I go on the website {string}', async (url) => { await browser.url(url) }) Then('should the title of the page be {string}', async (expectedTitle) => { assert.equal(await browser.getTitle(), expectedTitle) }) ``` **features/my-feature.feature** ```yaml Feature: Example feature As a user of Google I should be able to go to Google and see its correct title Scenario: Get title of website Given I go on the website "https://www.google.com/" Then should the title of the page be "Google" ``` Now we can run this test on TestingBot! To start the test, please run this command: ```bash npx wdio run wdio.conf.js ``` The test will now run on TestingBot. At the end of the test, the testname and success state will be available on TestingBot, together with a video and other meta-data. ## Mocha Example To run Mocha tests on TestingBot, please follow these steps: ```bash npm install @wdio/testingbot-service --save-dev npm install @wdio/dot-reporter --save-dev npm install @wdio/mocha-framework --save-dev ``` Now we'll make a simple Mocha test which uses Firefox on TestingBot to go to Google.com and verify the page's title. Please create the following files: **wdio.conf.js** ```javascript exports.config = { /** * specify test files */ specs: [ './runner-specs/mocha.test.js' ], /** * capabilities */ capabilities: [{ browserName: 'chrome', browserVersion: 'latest', platformName: 'WIN11' }], /** * test configurations */ logLevel: 'silent', waitforTimeout: 10000, connectionRetryTimeout: 480000, framework: 'mocha', services: [ ['testingbot'] ], user: 'API_KEY', key: 'API_SECRET', reporters: ['dot'], mochaOpts: { ui: 'bdd' } }; ``` **runner-specs/mocha.test.js** ```javascript const assert = require('assert'); describe('google page', async function() { it('should have the right title', async function () { await browser.url('https://www.google.com'); const title = await browser.getTitle(); assert.equal(title, 'Google'); }); }); ``` Now we can run this test on TestingBot. To start the test, please run this command: ```bash npx wdio run wdio.conf.js ``` The test will now run on TestingBot. At the end of the test, the testname and success state will be available on TestingBot, together with a video and other meta-data. ## Jasmine Example To run Jasmine tests on TestingBot, please follow these steps: ```bash npm install @wdio/testingbot-service --save-dev npm install @wdio/dot-reporter --save-dev npm install @wdio/jasmine-framework --save-dev ``` Now we'll make a simple Jasmine test which uses Firefox on TestingBot to go to Google.com and verify the page's title. Please create the following files: **wdio.conf.js** ```javascript exports.config = { /** * specify test files */ specs: [ './runner-specs/jasmine.spec.js' ], /** * capabilities */ capabilities: [{ browserName: 'chrome', browserVersion: 'latest', platformName: 'WIN11' }], /** * test configurations */ logLevel: 'silent', waitforTimeout: 10000, connectionRetryTimeout: 480000, framework: 'jasmine', services: [ ['testingbot'] ], user: 'API_KEY', key: 'API_SECRET', reporters: ['dot'], jasmineOpts: { defaultTimeoutInterval: 9999999 } }; ``` **runner-specs/jasmine.spec.js** ```javascript describe('google page', function() { it('should have the right title', async function () { await browser.url('https://www.google.com'); const title = await browser.getTitle(); expect(title).toBe('Google'); }); }); ``` Now we can run this test on TestingBot. To start the test, please run this command: ```bash npx wdio run wdio.conf.js ``` The test will now run on TestingBot. At the end of the test, the testname and success state will be available on TestingBot, together with a video and other meta-data. ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. When running mobile tests with WebdriverIO, especially on real devices, make sure to specify a large enough `connectionRetryTimeout` in case a device is in use: ```javascript connectionRetryTimeout: 480000 ``` ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a WebDriverIO test with our Tunnel: 1. Adjust your `wdio.conf.js` configuration to use `tbTunnel: true`. The [wdio-testingbot-service](https://webdriver.io/docs/testingbot-service/) will automatically download and use the tunnel for your tests. **wdio.conf.js** ```javascript exports.config = { services: [ ['testingbot', { tbTunnel: true }] ], user: 'API_KEY', key: 'API_SECRET' }; ``` ## Run Tests in Parallel Parallel testing allows you to run multiple tests at the same time, significantly reducing your total test execution time. You can run the same test across different browser configurations, or run different tests on the same configuration, all simultaneously. TestingBot provides a large grid of browsers and devices, enabling you to run tests in parallel efficiently. This is one of our core features designed to help you test faster and ship more reliably. With WebDriverIO, you can enable parallel test execution by setting the `maxInstances` property in your `wdio.conf.js` file and specifying multiple `capabilities`. This defines how many instances of the same browser can run in parallel. We recommend setting this value to match the number of parallel sessions allowed by your TestingBot plan. ```javascript exports.config = { maxInstances: 5, // Adjust this based on your TestingBot plan capabilities: [{ browserName: 'chrome', browserVersion: 'latest', platformName: 'WIN11' }, { browserName: 'firefox', browserVersion: 'latest', platformName: 'WIN11' }] }; ``` ### Queuing Every TestingBot plan includes a fixed number of parallel test slots. If you exceed this limit, additional tests will be automatically queued and executed as soon as slots become available, no tests will be dropped or skipped. ## Mark tests as passed/failed As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area. If you are using the [wdio-testingbot-service](https://webdriver.io/docs/testingbot-service/) then your tests will automatically report back meta-data to TestingBot (like test success, name, stacktrace, ...) ## Common errors Below are some common errors you might encounter when running WebDriverIO tests on TestingBot: - `UND_ERR_HEADERS_TIMEOUT` **Problem:** This error indicates that WebdriverIO did not receive a response in time. This might happen when requesting a device that is currently in use, or in case you are trying to run more tests in parallel than your plan allows and the tests have been [queued](https://testingbot.com#queuing) for a long time. **Solution:** You can try adding the `connectionRetryTimeout` in your WebDriverIO configuration with a high enough value: ```javascript connectionRetryTimeout: 480000 ``` ## Other NodeJS Framework examples - [CodeceptJS](https://testingbot.com/support/web-automate/selenium/nodejs/codeceptjs) Run acceptance tests with CodeceptJS on TestingBot. - [Nightwatch](https://testingbot.com/support/web-automate/selenium/nodejs/nightwatch) Nightwatch is an automated testing framework written in NodeJS. - [TestCafe](https://testingbot.com/support/web-automate/selenium/nodejs/testcafe) TestCafe is a NodeJS framework for end-to-end web testing. - [Hermione](https://testingbot.com/support/web-automate/selenium/nodejs/hermione) Hermione is a Test Framework similar to WebDriverIO, with automatic test retries and plugins. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/nodejs/jest # Jest Testing See the TestingBot [Jest example repository](https://github.com/testingbot/jest-js-testingbot-example) for an example on how to run Jest tests with remote TestingBot browsers. [Jest](https://jestjs.io/) is a popular open-source testing framework for JavaScript applications, developed and maintained by Facebook. It is widely used for testing React applications but is versatile enough to test any JavaScript codebase. Jest provides a simple, intuitive API for writing unit tests, integration tests and end-to-end tests. ## Setting up Jest To run Jest tests with WebDriver, you first have to make sure both the Jest NPM package and the Selenium WebDriver NPM package are added to your project. ```bash npm install --save-dev jest selenium-webdriver ``` You can now add a `jest.config.js` configuration file, which consists of a few options that Jest will use. ```javascript module.exports = { testEnvironment: 'node', testTimeout: 60000, // Extend timeout for Selenium tests coverageProvider: "v8", maxConcurrency: 5, maxWorkers: 5, testPathIgnorePatterns: ["/node_modules/"] }; ``` ### Jest Tests Now we can create our first Jest test. We'll create a new file in a directory called ` __tests__ `, called `single.test.js` ```javascript const { Builder, By } = require('selenium-webdriver'); const chrome = require('selenium-webdriver/chrome'); describe("TestingBot example test", () => { let driver; beforeAll(async () => { let options = new chrome.Options(); options.setBrowserVersion('latest'); options.setPlatform('WIN11'); options.set('tb:options', { 'key': process.env.TESTINGBOT_KEY, 'secret': process.env.TESTINGBOT_SECRET, 'name': 'Jest Sample Test' }); driver = await new Builder() .usingServer('https://hub.testingbot.com/wd/hub') .setChromeOptions(options) .build(); }); afterAll(async () => { if (driver) { await driver.quit(); } }); test("example test", async () => { await driver.get("https://testingbot.com"); await driver.findElement(By.css("a.button.button-trial")).click(); expect(await driver.getCurrentUrl()).toBe( "https://testingbot.com/users/sign_up" ); }); }); ``` This example test will run on a remote Chrome browser on Windows 11 in the TestingBot cloud. The browser will navigate to the TestingBot website, click a button and verify the URL. ### Run Jest test Now that we have a test, we can run it on TestingBot. Before we can do this, make sure you get your key and secret from the TestingBot dashboard, to authenticate with the remote browser grid. Now we can run the test: ```bash TESTINGBOT_KEY=... TESTINGBOT_SECRET=... npx jest ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a Jest test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```javascript driver = await new Builder() .usingServer('http://localhost:4445/wd/hub') .setChromeOptions(options) .build(); ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. By default, Jest will run all tests simultaneously. You can use Jest's [Sharding option](https://jestjs.io/docs/cli#--shard) to group tests together. For example, to split the suite into five shards, each running one fifth of the tests: ```javascript jest --shard=1/5 jest --shard=2/5 jest --shard=3/5 jest --shard=4/5 jest --shard=5/5 ``` ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Other NodeJS Framework examples - [WebDriverIO](https://testingbot.com/support/web-automate/selenium/nodejs/webdriverio) With WebDriverIO you can run Mocha, Jasmine and Cucumber tests. - [CodeceptJS](https://testingbot.com/support/web-automate/selenium/nodejs/codeceptjs) Run acceptance tests with CodeceptJS on TestingBot. - [Protractor](https://testingbot.com/support/web-automate/selenium/nodejs/protractor) Protractor is an end-to-end test framework for AngularJS applications. Protractor is a NodeJS program built on top of WebDriverJS. - [Soda](https://testingbot.com/support/web-automate/selenium/nodejs/soda) Selenium Node Adapter. A light-weight Selenium RC client for NodeJS. - [Nightwatch](https://testingbot.com/support/web-automate/selenium/nodejs/nightwatch) Nightwatch is an automated testing framework written in NodeJS. - [WD.js](https://testingbot.com/support/web-automate/selenium/nodejs/wd) WD.js is a NodeJS client for WebDriver/Selenium. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/nodejs/testcafe # TestCafe TestCafe is a NodeJS framework to automate end-to-end web testing. You can write tests in either JavaScript or TypeScript and is very easy to install and use. To get started, make sure you have: - NodeJS installed - Java (8 or higher) installed, only if you plan to start the [TestingBot Tunnel](https://testingbot.com/support/tunnel) jar locally. ## Install TestCafe To install TestCafe and the TestingBot plugin, please use this command: ```bash npm install -g testcafe testcafe-browser-provider-testingbot ``` Once this is installed, you can start using TestCafe to run tests on TestingBot. ## Configure the TestCafe + TestingBot plugin Please add these 2 environment variables to authenticate to our browser grid: ```bash export TB_KEY=YOUR-TESTINGBOT-KEY export TB_SECRET=YOUR-TESTINGBOT-SECRET ``` These two environment variables are required, however the plugin can be further configured with these environment variables as well: - `TB_TEST_NAME`: the text that will be displayed as Test Name in the dashboard. - `TB_BUILD`: you can add the tests as part of a build. - `SELENIUM_CAPABILITIES`: path to a file which contains a JSON formatted object with the capabilities you want to use. By default it will look for `capabilities.json`. - `TB_SCREEN_RESOLUTION`: set the VMs to a specific screen resolution. ## Test Example Please see this sample test, which you can use to run a test on our browser grid: ```javascript import { Selector } from 'testcafe'; fixture `Getting Started` .page `http://devexpress.github.io/testcafe/example`; test('My first test', async t => { await t .typeText('#developer-name', 'John Smith') .click('#submit-button') // Use the assertion to check if the actual header text is equal to the expected one .expect(Selector('#article-header').innerText).eql('Thank you, John Smith!'); }); ``` Now you can run this test on TestingBot: ```bash testcafe "testingbot:chrome@latest:WIN11" 'example.js' ``` TestCafe will start a [TestingBot Tunnel](https://testingbot.com/support/tunnel), then start a new Windows 11 VM on TestingBot with Chrome and run the test. You will see a video recording of the test in the member dashboard, together with screenshots and logs. ## Specify Browsers & Devices With our TestCafe plugin, you can use a file called [capabilities.json](https://github.com/testingbot/testcafe-browser-provider-testingbot/blob/master/capabilities.json) where you can specify a JSON object with capabilities for each browser you want to use. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. To run multiple tests simultaneously, you can specify multiple browsers, like this: ```bash testcafe "testingbot:chrome@latest:WIN11","testingbot:firefox@latest:SONOMA" 'path/to/test/file.js' ``` ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Other NodeJS Framework examples - [WebDriverIO](https://testingbot.com/support/web-automate/selenium/nodejs/webdriverio) With WebDriverIO you can run Mocha, Jasmine and Cucumber tests. - [CodeceptJS](https://testingbot.com/support/web-automate/selenium/nodejs/codeceptjs) Run acceptance tests with CodeceptJS on TestingBot. - [Nightwatch](https://testingbot.com/support/web-automate/selenium/nodejs/nightwatch) Nightwatch is an automated testing framework written in NodeJS. - [Protractor](https://testingbot.com/support/web-automate/selenium/nodejs/protractor) Protractor is an end-to-end test framework for AngularJS applications. - [WD.js](https://testingbot.com/support/web-automate/selenium/nodejs/wd) WD.js is a NodeJS client for WebDriver/Selenium. - [Hermione](https://testingbot.com/support/web-automate/selenium/nodejs/hermione) Hermione is a Test Framework similar to WebDriverIO, with automatic test retries and plugins. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/csharp # C# Automated Testing With TestingBot you can easily run your automated tests with [any C# test framework](https://testingbot.com#list). [Command Line](https://testingbot.com#) [Visual Studio](https://testingbot.com#) TestingBot has created a sample Visual Studio project to help you get started with C# and .NET Selenium testing. [Download ZIP](https://github.com/testingbot/nunit-example/archive/refs/heads/master.zip) You can import this project into Visual Studio and run the tests directly. To run your tests with C# and .NET, you need to have the [.NET SDK](https://dotnet.microsoft.com/download) installed on your machine. - **On Windows:** - Download the [.NET SDK](https://dotnet.microsoft.com/en-us/download) installer and run it. - Or use the `winget` command: `winget install Microsoft.DotNet.SDK.9` - **On MacOS:** `brew install dotnet` - **On Linux:** `sudo apt-get install dotnet-sdk-9.0` After installing the .NET SDK, you can create a new NUnit test project using the command line: ```bash dotnet new nunit -n SeleniumTest cd SeleniumTest ``` Next, add the Selenium WebDriver packages to your project: ```bash dotnet add package Selenium.WebDriver dotnet add package Selenium.Support ``` Replace the contents of `UnitTest1.cs` with the example code below, then run your tests with: ```bash TESTINGBOT_KEY=... TESTINGBOT_SECRET=... dotnet test ``` **Requirements:** The examples on this page require .NET 6.0 or higher (recommended: .NET 8.0+). ## Selenium C# Example The example below shows how to run a simple Selenium test using C# on TestingBot. It will instruct a remote browser to navigate to Google, mark the test as passed and then close the remote browser. ```csharp using NUnit.Framework; using NUnit.Framework.Interfaces; using OpenQA.Selenium; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Remote; using System; using System.Collections.Generic; namespace SeleniumNunit { [TestFixture] public class ExampleTest { private IWebDriver? Driver; [Test] public void SampleTest() { ChromeOptions options = new ChromeOptions(); options.BrowserVersion = "latest"; options.PlatformName = "WIN11"; options.AddAdditionalOption("tb:options", new Dictionary { ["key"] = Environment.GetEnvironmentVariable("TESTINGBOT_KEY") ?? "", ["secret"] = Environment.GetEnvironmentVariable("TESTINGBOT_SECRET") ?? "", ["name"] = TestContext.CurrentContext.Test.Name }); Driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), options); Driver.Navigate().GoToUrl("https://www.google.com"); Assert.Pass(); } [TearDown] public void CleanUpAfterEveryTestMethod() { if (Driver != null) { try { var passed = TestContext.CurrentContext.Result.Outcome.Status == TestStatus.Passed; var sessionId = ((RemoteWebDriver)Driver).SessionId.ToString(); using var client = new System.Net.Http.HttpClient(); client.Timeout = TimeSpan.FromSeconds(30); var key = Environment.GetEnvironmentVariable("TESTINGBOT_KEY"); var secret = Environment.GetEnvironmentVariable("TESTINGBOT_SECRET"); var auth = Convert.ToBase64String(System.Text.Encoding.ASCII.GetBytes($"{key}:{secret}")); client.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Basic", auth); var content = new System.Net.Http.FormUrlEncodedContent(new[] { new KeyValuePair("test[success]", passed ? "1" : "0") }); client.PutAsync($"https://api.testingbot.com/v1/tests/{sessionId}", content).Wait(); } catch (Exception) { // Ignore reporting errors so they don't fail an otherwise passing test } finally { Driver.Quit(); Driver.Dispose(); } } } } } ``` Depending on the browser you want to test on, you may want to use [a different DriverOptions class](https://www.selenium.dev/documentation/webdriver/drivers/options/): browserName | className || Safari | `OpenQA.Selenium.Safari.SafariOptions` | | Chrome | `OpenQA.Selenium.Chrome.ChromeOptions` | | Firefox | `OpenQA.Selenium.Firefox.FirefoxOptions` | | IE | `OpenQA.Selenium.IE.InternetExplorerOptions` | | Edge | `OpenQA.Selenium.Edge.EdgeOptions` | Make sure to always stop your test (`Driver.Quit()`), otherwise it will continue running, leading to a timeout. ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a C# test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```csharp using NUnit.Framework; using NUnit.Framework.Interfaces; using OpenQA.Selenium; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Remote; using System; using System.Collections.Generic; namespace SeleniumNunit { [TestFixture] public class ExampleTest { private IWebDriver? Driver; [Test] public void SampleTest() { ChromeOptions options = new ChromeOptions(); options.BrowserVersion = "latest"; options.PlatformName = "WIN11"; options.AddAdditionalOption("tb:options", new Dictionary { ["key"] = Environment.GetEnvironmentVariable("TESTINGBOT_KEY") ?? "", ["secret"] = Environment.GetEnvironmentVariable("TESTINGBOT_SECRET") ?? "", ["name"] = TestContext.CurrentContext.Test.Name }); Driver = new RemoteWebDriver(new Uri("http://localhost:4445/wd/hub"), options); Driver.Navigate().GoToUrl("https://www.google.com"); Assert.Pass(); } [TearDown] public void CleanUpAfterEveryTestMethod() { if (Driver != null) { try { var passed = TestContext.CurrentContext.Result.Outcome.Status == TestStatus.Passed; var sessionId = ((RemoteWebDriver)Driver).SessionId.ToString(); using var client = new System.Net.Http.HttpClient(); client.Timeout = TimeSpan.FromSeconds(30); var key = Environment.GetEnvironmentVariable("TESTINGBOT_KEY"); var secret = Environment.GetEnvironmentVariable("TESTINGBOT_SECRET"); var auth = Convert.ToBase64String(System.Text.Encoding.ASCII.GetBytes($"{key}:{secret}")); client.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Basic", auth); var content = new System.Net.Http.FormUrlEncodedContent(new[] { new KeyValuePair("test[success]", passed ? "1" : "0") }); client.PutAsync($"https://api.testingbot.com/v1/tests/{sessionId}", content).Wait(); } catch (Exception) { // Ignore reporting errors so they don't fail an otherwise passing test } finally { Driver.Quit(); Driver.Dispose(); } } } } } ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. ### NUnit Parallel Testing NUnit supports parallel test execution out of the box using the `[Parallelizable]` attribute: ```csharp // Run all tests in this class in parallel with other classes [TestFixture] [Parallelizable(ParallelScope.All)] public class ParallelTests { [Test] public void Test1() { /* ... */ } [Test] public void Test2() { /* ... */ } } // Or at assembly level in AssemblyInfo.cs: [assembly: Parallelizable(ParallelScope.Fixtures)] [assembly: LevelOfParallelism(4)] // Run up to 4 tests concurrently ``` You can also run tests in parallel from the command line: ```bash dotnet test -- NUnit.NumberOfTestWorkers=4 ``` ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed To see if a test passed or not in our member area, or to send additional meta-data to TestingBot, you can use our API. Please see the example below on how to notify TestingBot about the test success state: ```csharp [TearDown] public void CleanUp() { if (Driver != null) { try { bool passed = TestContext.CurrentContext.Result.Outcome.Status == TestStatus.Passed; // Logs the result to TestingBot via the executeScript hook ((IJavaScriptExecutor)Driver).ExecuteScript("tb:test-result=" + (passed ? "passed" : "failed")); } finally { Driver.Quit(); Driver.Dispose(); } } } ``` ## Other C# Framework examples - [NUnit](https://testingbot.com/support/web-automate/selenium/csharp/nunit) An unit testing framework that is open source written in C#. - [PNunit](https://testingbot.com/support/web-automate/selenium/csharp/pnunit) With PNUnit you can run several tests in parallel. - [SpecFlow](https://testingbot.com/support/web-automate/selenium/csharp/specflow) SpecFlow allows you to run Automated .NET tests using Cucumber-compatible Gherkin syntax. - [MSTest](https://testingbot.com/support/web-automate/selenium/csharp/mstest) MSTest framework is a test framework which is included, by default, with Microsoft Visual Studio. - [xUnit](https://testingbot.com/support/web-automate/selenium/csharp/xunit) xUnit.net is a modern, extensible testing framework for .NET, widely used in the .NET ecosystem. - [Reqnroll](https://testingbot.com/support/web-automate/selenium/csharp/reqnroll) Reqnroll is the successor to SpecFlow for BDD testing with Gherkin syntax. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/csharp/pnunit **PNUnit is legacy and unmaintained.** It shipped with NUnit 2.x (the `pNUnit` / `NUnit.Runners` 2.6.4 packages) and its `pnunit-agent` / `pnunit-launcher` tooling targets .NET Framework on Windows only — it relies on .NET Remoting and does not run on .NET Core / .NET 5+ or on Linux/macOS. NUnit 3+ removed PNUnit entirely. For parallel testing on modern .NET, use the [NUnit parallel option](https://testingbot.com/support/web-automate/selenium/csharp/nunit#parallel) (`[Parallelizable]` / `LevelOfParallelism`) instead. # PNUnit Automated Testing PNUnit is an expansion to the NUnit framework. The "P" stands for "Parallel" and basically means that you can run several tests at the same time on our Selenium grid. More information regarding PNUnit is available on the [PNUnit website](https://www.plasticscm.com/documentation/technical-articles/pnunit-parallel-nunit). It's really easy to convert your current NUnit Selenium tests to PNUnit Selenium Tests. - Modify your test to take parameters from a config file (browserName, version, platform, ...) - Create the config file (shown in the example below), specifying which browsers you want to test on. ## Example code ```csharp using NUnit.Framework; using PNUnit.Framework; using System; using System.Collections.Generic; using OpenQA.Selenium; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Remote; namespace Test { [TestFixture] public class TestingBotTest { private IWebDriver driver; private string[] testParams; [SetUp] public void Init() { testParams = PNUnitServices.Get().GetTestParams(); string browser = testParams[0]; string version = testParams[1]; string platform = testParams[2]; ChromeOptions options = new ChromeOptions(); options.BrowserVersion = version; options.PlatformName = platform; var tbOptions = new Dictionary { ["key"] = "API_KEY", ["secret"] = "API_SECRET" }; options.AddAdditionalOption("tb:options", tbOptions); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), options); } [Test] public void TestCase() { driver.Navigate().GoToUrl("https://www.google.com"); StringAssert.Contains("Google", driver.Title); IWebElement query = driver.FindElement(By.Name("q")); query.SendKeys("TestingBot"); query.Submit(); } [TearDown] public void CleanUp() { bool passed = TestContext.CurrentContext.Result.Outcome.Status == NUnit.Framework.Interfaces.TestStatus.Passed; try { // Logs the result to TestingBot ((IJavaScriptExecutor)driver).ExecuteScript("tb:test-result=" + (passed ? "passed" : "failed")); } finally { driver.Quit(); } } } } ``` Now we have to tell PNUnit which parameters we want to send with the test: ```xml Testing TestFF-Win11 SeleniumTest.dll Test.TestingBotTest.TestCase localhost:8080 firefox latest WIN11 TestChrome-Win11 SeleniumTest.dll Test.TestingBotTest.TestCase localhost:8080 chrome latest WIN11 ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ```csharp ChromeOptions options = new ChromeOptions(); options.BrowserVersion = version; options.PlatformName = platform; var tbOptions = new Dictionary { ["key"] = "API_KEY", ["secret"] = "API_SECRET" }; options.AddAdditionalOption("tb:options", tbOptions); ``` To see how to do this, please select a combination of browser, version and platform in the drop-down menus below. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a PNUnit test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```csharp using NUnit.Framework; using PNUnit.Framework; using System; using System.Collections.Generic; using OpenQA.Selenium; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Remote; namespace Test { [TestFixture] public class TestingBotTest { private IWebDriver driver; private string[] testParams; [SetUp] public void Init() { testParams = PNUnitServices.Get().GetTestParams(); string browser = testParams[0]; string version = testParams[1]; string platform = testParams[2]; ChromeOptions options = new ChromeOptions(); options.BrowserVersion = version; options.PlatformName = platform; var tbOptions = new Dictionary { ["key"] = "API_KEY", ["secret"] = "API_SECRET" }; options.AddAdditionalOption("tb:options", tbOptions); // Point to the tunnel running on localhost:4445 driver = new RemoteWebDriver(new Uri("http://localhost:4445/wd/hub"), options); } [Test] public void TestCase() { driver.Navigate().GoToUrl("https://www.google.com"); StringAssert.Contains("Google", driver.Title); IWebElement query = driver.FindElement(By.Name("q")); query.SendKeys("TestingBot"); query.Submit(); } [TearDown] public void CleanUp() { bool passed = TestContext.CurrentContext.Result.Outcome.Status == NUnit.Framework.Interfaces.TestStatus.Passed; try { // Logs the result to TestingBot ((IJavaScriptExecutor)driver).ExecuteScript("tb:test-result=" + (passed ? "passed" : "failed")); } finally { driver.Quit(); } } } } ``` ## Mark tests as passed/failed To see if a test passed or not in our member area, or to send additional meta-data to TestingBot, you can use our API. Please see the example below on how to notify TestingBot about the test success state: ```csharp [TearDown] public void CleanUp() { bool passed = TestContext.CurrentContext.Result.Outcome.Status == NUnit.Framework.Interfaces.TestStatus.Passed; try { // Logs the result to TestingBot ((IJavaScriptExecutor)driver).ExecuteScript("tb:test-result=" + (passed ? "passed" : "failed")); } finally { driver.Quit(); } } ``` ## Other C# Framework examples - [NUnit](https://testingbot.com/support/web-automate/selenium/csharp/nunit) A unit testing framework that is open source written in C#. - [PNunit](https://testingbot.com/support/web-automate/selenium/csharp/pnunit) With PNUnit you can run several tests in parallel. - [SpecFlow](https://testingbot.com/support/web-automate/selenium/csharp/specflow) SpecFlow allows you to run Automated .NET tests using Cucumber-compatible Gherkin syntax. - [MSTest](https://testingbot.com/support/web-automate/selenium/csharp/mstest) MSTest framework is a test framework which is included, by default, with Microsoft Visual Studio. - [xUnit](https://testingbot.com/support/web-automate/selenium/csharp/xunit) xUnit.net is a modern, extensible testing framework for .NET, widely used in the .NET ecosystem. - [Reqnroll](https://testingbot.com/support/web-automate/selenium/csharp/reqnroll) Reqnroll is the successor to SpecFlow for BDD testing with Gherkin syntax. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/csharp/nunit # C# Automated Testing with NUnit NUnit is a unit-test framework designed for all the .NET languages. It is written in C# and built to use many of the .NET language features. For example: it allows you to use custom attributes and other reflection-related capabilities. Please visit the [official NUnit website](https://nunit.org/) for more documentation about the framework. See our [NUnit example repository](https://github.com/testingbot/nunit-example) for a simple example on how to run NUnit tests on TestingBot. ## Prerequisites To get started, you need to download and install these components: [Command Line](https://testingbot.com#) [Visual Studio](https://testingbot.com#) TestingBot has created a sample [Visual Studio](https://www.visualstudio.com/) project to help you get started with C# and .NET Selenium testing. [Download ZIP](https://github.com/testingbot/nunit-example/archive/refs/heads/master.zip) You can import this project into Visual Studio and run the tests directly. To run your tests with C# and .NET, you need to have the [.NET SDK](https://dotnet.microsoft.com/download) installed on your machine. - **On Windows:** - Download the [.NET SDK](https://dotnet.microsoft.com/en-us/download) installer and run it. - Or use the `winget` command: `winget install Microsoft.DotNet.SDK.9` - **On MacOS:** `brew install dotnet` - **On Linux:** `sudo apt-get install dotnet-sdk-9.0` After installing the .NET SDK, you can create a new C# project using the command line: ```bash dotnet new nunit -n SeleniumTest ``` The `nunit` template already includes NUnit, the test SDK and the NUnit test adapter (required for `dotnet test` to discover and run your tests). Next, add the Selenium WebDriver packages to your project: ```bash cd SeleniumTest dotnet add package Selenium.WebDriver dotnet add package Selenium.Support ``` ## Example NUnit test Below is a simple example of an NUnit test that uses Selenium WebDriver to open Google and search for "TestingBot". ```csharp using OpenQA.Selenium; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Firefox; using OpenQA.Selenium.Remote; using OpenQA.Selenium.Safari; namespace NUnit_TestingBot_Sample; [TestFixture("chrome", "latest", "WIN11")] public class TbNUnit_Test { private IWebDriver driver; private string browser; private string version; private string os; public TbNUnit_Test(string browser, string version, string os) { this.browser = browser; this.version = version; this.os = os; } [SetUp] public void Init() { var browserOptions = new ChromeOptions { BrowserVersion = this.version, PlatformName = this.os }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["name"] = TestContext.CurrentContext.Test.Name }; browserOptions.AddAdditionalOption("tb:options", tbOptions); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), browserOptions); } [Test] public void GoogleTest() { driver.Navigate().GoToUrl("https://www.google.com"); Assert.That(driver.Title, Is.EqualTo("Google")); IWebElement query = driver.FindElement(By.Name("q")); query.SendKeys("TestingBot"); query.Submit(); } [TearDown] public void CleanUp() { bool passed = TestContext.CurrentContext.Result.Outcome.Status == NUnit.Framework.Interfaces.TestStatus.Passed; var sessionId = ((RemoteWebDriver)driver).SessionId.ToString(); try { // Report the result to TestingBot via the REST API using var client = new System.Net.Http.HttpClient(); client.Timeout = TimeSpan.FromSeconds(30); var key = "api_key"; var secret = "api_secret"; var auth = Convert.ToBase64String(System.Text.Encoding.ASCII.GetBytes($"{key}:{secret}")); client.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Basic", auth); var content = new System.Net.Http.FormUrlEncodedContent(new[] { new KeyValuePair("test[success]", passed ? "1" : "0") }); client.PutAsync($"https://api.testingbot.com/v1/tests/{sessionId}", content).Wait(); } catch (Exception) { // Ignore reporting errors so they don't fail an otherwise passing test } finally { driver.Quit(); driver.Dispose(); } } } ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "WIN11" }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["name"] = TestContext.CurrentContext.Test.Name }; ``` Depending on the browser you want to test on, you might need to specify different browser options in your code: - [ChromeOptions](https://github.com/SeleniumHQ/selenium/blob/trunk/dotnet/src/webdriver/Chrome/ChromeOptions.cs) for Chrome browsers - [EdgeOptions](https://learn.microsoft.com/en-us/microsoft-edge/webdriver-chromium/capabilities-edge-options) for Edge browsers - [FirefoxOptions](https://github.com/SeleniumHQ/selenium/blob/trunk/dotnet/src/webdriver/Firefox/FirefoxOptions.cs) for Firefox browsers - [InternetExplorerOptions](https://learn.microsoft.com/en-us/microsoft-edge/webdriver-chromium/ie-mode?tabs=c-sharp) for IE browsers - [SafariOptions](https://github.com/SeleniumHQ/selenium/blob/trunk/dotnet/src/webdriver/Safari/SafariOptions.cs) for Safari browsers ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. Looking to generate more custom capabilities? Use the [Selenium capabilities generator](https://testingbot.com/support/web-automate/selenium/capabilities). ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a C# test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```csharp using OpenQA.Selenium; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Remote; namespace NUnit_TestingBot_Sample; [TestFixture("chrome", "latest", "WIN11")] public class TbNUnit_Tunnel_Test { private IWebDriver driver; private string browser; private string version; private string os; public TbNUnit_Tunnel_Test(string browser, string version, string os) { this.browser = browser; this.version = version; this.os = os; } [SetUp] public void Init() { var browserOptions = new ChromeOptions { BrowserVersion = this.version, PlatformName = this.os }; var tbOptions = new Dictionary { ["name"] = TestContext.CurrentContext.Test.Name }; browserOptions.AddAdditionalOption("tb:options", tbOptions); driver = new RemoteWebDriver(new Uri("http://localhost:4445/wd/hub"), browserOptions); } [Test] public void GoogleTest() { driver.Navigate().GoToUrl("https://www.google.com"); Assert.That(driver.Title, Is.EqualTo("Google")); } [TearDown] public void CleanUp() { driver.Quit(); driver.Dispose(); } } ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. NUnit provides a `[assembly: Parallelizable(ParallelScope.Fixtures)]` option to specify tests should run in parallel. More information is available in the [NUnit Parallelizable documentation](https://docs.nunit.org/articles/nunit/technical-notes/usage/Framework-Parallel-Test-Execution.html#specifying-parallelism-at-multiple-test-levels). ### Queuing Every plan we provide comes with a limited amount of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed To see if a test passed or not in our member area, or to send additional meta-data to TestingBot, you can use our API. Please see the example below on how to notify TestingBot about the test success state: ```csharp [TearDown] public void CleanUp() { bool passed = TestContext.CurrentContext.Result.Outcome.Status == NUnit.Framework.Interfaces.TestStatus.Passed; try { // Logs the result to TestingBot ((IJavaScriptExecutor)driver).ExecuteScript("tb:test-result=" + (passed ? "passed" : "failed")); } finally { // Terminates the remote webdriver session driver.Quit(); driver.Dispose(); } } ``` ## Other C# Framework examples - [NUnit](https://testingbot.com/support/web-automate/selenium/csharp/nunit) An unit testing framework that is open source written in C#. - [PNunit](https://testingbot.com/support/web-automate/selenium/csharp/pnunit) With PNUnit you can run several tests in parallel. - [SpecFlow](https://testingbot.com/support/web-automate/selenium/csharp/specflow) SpecFlow allows you to run Automated .NET tests using Cucumber-compatible Gherkin syntax. - [MSTest](https://testingbot.com/support/web-automate/selenium/csharp/mstest) MSTest framework is a test framework which is included, by default, with Microsoft Visual Studio. - [xUnit](https://testingbot.com/support/web-automate/selenium/csharp/xunit) xUnit.net is a modern, extensible testing framework for .NET, widely used in the .NET ecosystem. - [Reqnroll](https://testingbot.com/support/web-automate/selenium/csharp/reqnroll) Reqnroll is the successor to SpecFlow for BDD testing with Gherkin syntax. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/csharp/mbunit # C# Automated Testing with MbUnit **MbUnit / Gallio is discontinued.** The last release dates from 2014 and only targets .NET Framework on Windows. It depends on .NET Remoting (removed from .NET Core / .NET 5+) and has no test runner for modern .NET, so these examples will not run on .NET 6/7/8 or on Linux/macOS. For new projects use [NUnit](https://testingbot.com/support/web-automate/selenium/csharp/nunit), [xUnit](https://testingbot.com/support/web-automate/selenium/csharp/xunit) or [MSTest](https://testingbot.com/support/web-automate/selenium/csharp/mstest) instead. MbUnit (previously named GUnit) is a generative test unit framework, built for C sharp testing. MbUnit implements the Simple Test Pattern and makes it really easy to use fixtures. ## Your first MbUnit test Please see the example test below, using MbUnit and Gallio framework: **SingleTest.cs** : ```csharp using Gallio.Framework; using MbUnit.Framework; using MbUnit.Framework.ContractVerifiers; using OpenQA.Selenium; using System.Collections.Generic; namespace TestingBot { [TestFixture] public class SingleTest : TestingBotMBUnitTest { public SingleTest() : base("single", "chrome") { } [Test] public void SearchGoogle() { driver.Navigate().GoToUrl("https://www.google.com/ncr"); IWebElement query = driver.FindElement(By.Name("q")); query.SendKeys("TestingBot"); query.Submit(); System.Threading.Thread.Sleep(5000); Assert.AreEqual("TestingBot - Google Search", driver.Title); } } } ``` **TestingBotMBUnitTest.cs** : ```csharp using Gallio.Framework; using MbUnit.Framework; using MbUnit.Framework.ContractVerifiers; using OpenQA.Selenium; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Remote; using System; using System.Collections.Generic; namespace TestingBot { [TestFixture] public class TestingBotMBUnitTest { protected IWebDriver driver; protected string profile; protected string environment; public TestingBotMBUnitTest(string profile, string environment = "chrome") { this.profile = profile; this.environment = environment; } [FixtureSetUp] public void Init() { string key = Environment.GetEnvironmentVariable("TESTINGBOT_KEY"); string secret = Environment.GetEnvironmentVariable("TESTINGBOT_SECRET"); ChromeOptions options = new ChromeOptions(); options.BrowserVersion = "latest"; options.PlatformName = "WIN11"; var tbOptions = new Dictionary { ["key"] = key, ["secret"] = secret, ["name"] = profile }; options.AddAdditionalOption("tb:options", tbOptions); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), options); } [FixtureTearDown] public void Cleanup() { driver.Quit(); } } } ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ```csharp ChromeOptions options = new ChromeOptions(); options.BrowserVersion = "latest"; options.PlatformName = "WIN11"; var tbOptions = new Dictionary { ["key"] = "API_KEY", ["secret"] = "API_SECRET" }; options.AddAdditionalOption("tb:options", tbOptions); IWebDriver driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), options ); ``` To see how to do this, please select a combination of browser, version and platform in the drop-down menus below. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a C# test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```csharp driver = new RemoteWebDriver(new Uri("http://localhost:4445/wd/hub"), options); ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. Please see our [PNUnit documentation](https://testingbot.com/support/web-automate/selenium/csharp/pnunit) for parallel testing. ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed To see if a test passed or not in our member area, or to send additional meta-data to TestingBot, you can use our API. Please see the example below on how to notify TestingBot about the test success state: ```csharp [FixtureTearDown] public void CleanUp() { bool passed = TestContext.CurrentContext.Outcome.Status == Gallio.Model.TestStatus.Passed; var sessionId = ((RemoteWebDriver)driver).SessionId.ToString(); try { // Report the result to TestingBot via the REST API using (var client = new System.Net.Http.HttpClient()) { var key = Environment.GetEnvironmentVariable("TESTINGBOT_KEY"); var secret = Environment.GetEnvironmentVariable("TESTINGBOT_SECRET"); var auth = Convert.ToBase64String(System.Text.Encoding.ASCII.GetBytes($"{key}:{secret}")); client.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Basic", auth); var content = new System.Net.Http.FormUrlEncodedContent(new[] { new KeyValuePair("test[success]", passed ? "1" : "0") }); client.PutAsync($"https://api.testingbot.com/v1/tests/{sessionId}", content).Wait(); } } finally { driver.Quit(); } } ``` ## Other C# Framework examples - [NUnit](https://testingbot.com/support/web-automate/selenium/csharp/nunit) An unit testing framework that is open source written in C#. - [PNunit](https://testingbot.com/support/web-automate/selenium/csharp/pnunit) With PNUnit you can run several tests in parallel. - [SpecFlow](https://testingbot.com/support/web-automate/selenium/csharp/specflow) SpecFlow allows you to run Automated .NET tests using Cucumber-compatible Gherkin syntax. - [MSTest](https://testingbot.com/support/web-automate/selenium/csharp/mstest) MSTest framework is a test framework which is included, by default, with Microsoft Visual Studio. - [MbUnit](https://testingbot.com/support/web-automate/selenium/csharp/mbunit) MbUnit is a generative test unit framework, built for C sharp testing. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/csharp/mstest # C# Automated Testing with MSTest MSTest framework is a test framework which is included, by default, with Microsoft Visual Studio. It is also referred to as Visual Studio Unit Testing Framework and is popular with developers using the Visual Studio IDE. MSTest V2 is open-sourced and available on [GitHub](https://github.com/Microsoft/testfx). With MSTest V2 you can easily run tests in parallel, using In-Assembly Parallel (via `Annotations` or `RunSettings`). ## Installation To get started with MSTest, you'll need to make sure that `MSTest Framework` and `MSTest Adapter` are installed. You can install the packages via the commandline or with NuGet: - MSTest.TestAdapter - MSTest.TestFramework - Microsoft.NET.Test.Sdk ## Your first MSTest test Please see the example test below, using MSTest. The test will start 3 browsers, visit our homepage and print the title. **SingleTest.cs** : ```csharp using Microsoft.VisualStudio.TestTools.UnitTesting; using System; using System.Collections.Generic; using OpenQA.Selenium; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Firefox; using OpenQA.Selenium.Edge; using OpenQA.Selenium.Remote; [assembly: Parallelize(Workers = 0, Scope = ExecutionScope.ClassLevel)] namespace MS_Test_Cross_Browser { [TestClass] public class SingleTest { IWebDriver driver; [DataTestMethod] [DataRow("chrome", "latest", "WIN11")] [DataRow("MicrosoftEdge", "latest-1", "WIN11")] [DataRow("firefox", "latest", "SONOMA")] public void Cross_Browser_Test(String browser, String version, String os) { DriverOptions options; switch (browser.ToLower()) { case "firefox": options = new FirefoxOptions(); break; case "microsoftedge": options = new EdgeOptions(); break; default: options = new ChromeOptions(); break; } options.BrowserVersion = version; options.PlatformName = os; options.AddAdditionalOption("tb:options", new Dictionary { ["key"] = "API_KEY", ["secret"] = "API_SECRET", ["name"] = "MSTest with TestingBot" }); driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), options ); driver.Navigate().GoToUrl("https://testingbot.com"); Console.WriteLine(driver.Title); } [TestCleanup] public void Cleanup() { driver?.Quit(); } } } ``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ```csharp ChromeOptions options = new ChromeOptions(); options.BrowserVersion = "latest"; options.PlatformName = "WIN11"; options.AddAdditionalOption("tb:options", new Dictionary { ["key"] = "API_KEY", ["secret"] = "API_SECRET", ["name"] = "My Test" }); IWebDriver driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), options ); ``` To see how to do this, please select a combination of browser, version and platform in the drop-down menus below. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a C# test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```csharp IWebDriver driver = new RemoteWebDriver(new Uri("http://localhost:4445/wd/hub"), options); ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. To modify the number of parallel tests with MSTest, please use the `Parallelize annotation`: ```csharp [assembly: Parallelize(Workers = 5, Scope = ExecutionScope.MethodLevel)] ``` This will run the test on 5 workers (= 5 parallel sessions). ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed To see if a test passed or not in our member area, or to send additional meta-data to TestingBot, you can use our API. Please see the example below on how to notify TestingBot about the test success state: ```csharp // MSTest injects the TestContext via this public property - it is required // to read the test outcome below, so make sure your test class declares it. public TestContext TestContext { get; set; } [TestCleanup] public void CleanUp() { bool passed = TestContext.CurrentTestOutcome == UnitTestOutcome.Passed; var sessionId = ((RemoteWebDriver)driver).SessionId.ToString(); try { // Report the result to TestingBot via the REST API using var client = new System.Net.Http.HttpClient(); client.Timeout = TimeSpan.FromSeconds(30); var key = Environment.GetEnvironmentVariable("TESTINGBOT_KEY"); var secret = Environment.GetEnvironmentVariable("TESTINGBOT_SECRET"); var auth = Convert.ToBase64String(System.Text.Encoding.ASCII.GetBytes($"{key}:{secret}")); client.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Basic", auth); var content = new System.Net.Http.FormUrlEncodedContent(new[] { new KeyValuePair("test[success]", passed ? "1" : "0") }); client.PutAsync($"https://api.testingbot.com/v1/tests/{sessionId}", content).Wait(); } catch (Exception) { // Ignore reporting errors so they don't fail an otherwise passing test } finally { driver.Quit(); } } ``` ## Other C# Framework examples - [NUnit](https://testingbot.com/support/web-automate/selenium/csharp/nunit) An unit testing framework that is open source written in C#. - [PNunit](https://testingbot.com/support/web-automate/selenium/csharp/pnunit) With PNUnit you can run several tests in parallel. - [SpecFlow](https://testingbot.com/support/web-automate/selenium/csharp/specflow) SpecFlow allows you to run Automated .NET tests using Cucumber-compatible Gherkin syntax. - [MSTest](https://testingbot.com/support/web-automate/selenium/csharp/mstest) MSTest framework is a test framework which is included, by default, with Microsoft Visual Studio. - [xUnit](https://testingbot.com/support/web-automate/selenium/csharp/xunit) xUnit.net is a modern, extensible testing framework for .NET, widely used in the .NET ecosystem. - [Reqnroll](https://testingbot.com/support/web-automate/selenium/csharp/reqnroll) Reqnroll is the successor to SpecFlow for BDD testing with Gherkin syntax. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/csharp/xunit # C# Automated Testing with xUnit.net [xUnit.net](https://xunit.net/) is a free, open-source, community-focused unit testing framework for .NET. It is the successor to NUnit and was written by the original inventor of NUnit v2. xUnit.net is widely used in the .NET ecosystem and offers modern features like: - Built-in support for parallel test execution - Theory tests with `[InlineData]` for data-driven testing - Constructor/Dispose pattern for test setup and teardown - Fixtures for sharing context between tests ## Installation Create a new xUnit test project and add the required packages: ```bash # Create a new xUnit test project dotnet new xunit -n SeleniumTests cd SeleniumTests # Add Selenium WebDriver packages dotnet add package Selenium.WebDriver dotnet add package Selenium.Support ``` Or add these packages to your `.csproj` file: ```xml ``` **Requirements:** xUnit 2.x requires .NET 6.0 or higher. The examples use modern C# features like nullable reference types. ## Your first xUnit test The example below shows a complete xUnit test that connects to TestingBot and runs a simple browser test. xUnit uses constructor/destructor (or `IDisposable`) for setup and teardown instead of attributes. **GoogleTest.cs** : ```csharp using System; using System.Collections.Generic; using OpenQA.Selenium; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Remote; using Xunit; namespace SeleniumTests; public class GoogleTest : IDisposable { private readonly IWebDriver _driver; private readonly string _sessionId; public GoogleTest() { // Configure Chrome options var options = new ChromeOptions(); options.BrowserVersion = "latest"; options.PlatformName = "WIN11"; options.AddAdditionalOption("tb:options", new Dictionary { ["key"] = Environment.GetEnvironmentVariable("TESTINGBOT_KEY") ?? "", ["secret"] = Environment.GetEnvironmentVariable("TESTINGBOT_SECRET") ?? "", ["name"] = "xUnit Google Test" }); // Connect to TestingBot _driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), options ); _sessionId = ((RemoteWebDriver)_driver).SessionId.ToString(); } [Fact] public void GoogleTitle_ShouldContainGoogle() { _driver.Navigate().GoToUrl("https://www.google.com"); Assert.Contains("Google", _driver.Title); } public void Dispose() { // Quit also disposes the underlying remote session _driver?.Quit(); } } ``` Run your test with: ```bash TESTINGBOT_KEY=your_key TESTINGBOT_SECRET=your_secret dotnet test ``` ## Data-Driven Testing with Theory xUnit's `[Theory]` attribute allows you to run the same test with different data. This is perfect for cross-browser testing: ```csharp using OpenQA.Selenium; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Firefox; using OpenQA.Selenium.Edge; using OpenQA.Selenium.Safari; using OpenQA.Selenium.Remote; using Xunit; namespace SeleniumTests; public class CrossBrowserTest : IDisposable { private IWebDriver? _driver; [Theory] [InlineData("chrome", "latest", "WIN11")] [InlineData("firefox", "latest", "WIN11")] [InlineData("MicrosoftEdge", "latest", "WIN11")] [InlineData("safari", "latest", "SONOMA")] public void HomePage_ShouldLoad_OnAllBrowsers(string browser, string version, string platform) { // Create browser-specific options DriverOptions options = browser.ToLower() switch { "chrome" => new ChromeOptions(), "firefox" => new FirefoxOptions(), "microsoftedge" => new EdgeOptions(), "safari" => new SafariOptions(), _ => new ChromeOptions() }; options.BrowserVersion = version; options.PlatformName = platform; options.AddAdditionalOption("tb:options", new Dictionary { ["key"] = Environment.GetEnvironmentVariable("TB_KEY") ?? "", ["secret"] = Environment.GetEnvironmentVariable("TB_SECRET") ?? "", ["name"] = $"Cross-browser test - {browser}" }); _driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), options ); _driver.Navigate().GoToUrl("https://testingbot.com"); Assert.Contains("TestingBot", _driver.Title); } public void Dispose() { _driver?.Quit(); } } ``` ## Specify Browsers & Devices To run tests on specific browsers and platforms, configure the appropriate `DriverOptions` class. ```csharp // Chrome on Windows var chromeOptions = new ChromeOptions(); chromeOptions.BrowserVersion = "latest"; chromeOptions.PlatformName = "WIN11"; chromeOptions.AddAdditionalOption("tb:options", new Dictionary { ["key"] = "API_KEY", ["secret"] = "API_SECRET" }); var driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions ); ``` To see how to configure different browsers and platforms, use the picker below: ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites Use [TestingBot Tunnel](https://testingbot.com/support/tunnel) to securely test internal or staged websites that aren't publicly accessible. **1.** [Download the tunnel](https://testingbot.com/support/tunnel) and start it: ```bash java -jar testingbot-tunnel.jar API_KEY API_SECRET ``` **2.** Update your test to connect through the tunnel (change the hub URL to `localhost:4445`): ```csharp // Connect through the tunnel instead of directly to TestingBot var driver = new RemoteWebDriver( new Uri("http://localhost:4445/wd/hub"), options ); // Now you can access internal URLs driver.Navigate().GoToUrl("http://internal.yourcompany.com"); ``` ## Run tests in Parallel xUnit runs tests in parallel by default. Tests in different test classes run in parallel, while tests in the same class run sequentially. ### Configure Parallelism Control parallel execution by adding an `xunit.runner.json` file to your project: ```json { "parallelizeAssembly": true, "parallelizeTestCollections": true, "maxParallelThreads": 5 } ``` Make sure to include it in your `.csproj`: ```xml ``` ### Using Test Collections Group tests that shouldn't run in parallel using `[Collection]`: ```csharp // Tests in the same collection run sequentially [Collection("Sequential")] public class SequentialTests1 { } [Collection("Sequential")] public class SequentialTests2 { } // Tests without a collection run in parallel with other classes ``` ### Queuing Every TestingBot plan has a limit on parallel tests. If you exceed this limit, additional tests are queued (up to 6 minutes) and run as slots become available. ## Mark tests as passed/failed To report test results to TestingBot, use JavaScript execution to set the test status: ```csharp public class TestBase : IDisposable { protected IWebDriver Driver { get; } private bool _testPassed = true; public TestBase() { var options = new ChromeOptions(); options.BrowserVersion = "latest"; options.PlatformName = "WIN11"; options.AddAdditionalOption("tb:options", new Dictionary { ["key"] = Environment.GetEnvironmentVariable("TESTINGBOT_KEY") ?? "", ["secret"] = Environment.GetEnvironmentVariable("TESTINGBOT_SECRET") ?? "" }); Driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), options ); } protected void MarkTestFailed() { _testPassed = false; } public void Dispose() { if (Driver != null) { try { var sessionId = ((RemoteWebDriver)Driver).SessionId.ToString(); using var client = new System.Net.Http.HttpClient(); client.Timeout = TimeSpan.FromSeconds(30); var key = Environment.GetEnvironmentVariable("TESTINGBOT_KEY"); var secret = Environment.GetEnvironmentVariable("TESTINGBOT_SECRET"); var auth = Convert.ToBase64String(System.Text.Encoding.ASCII.GetBytes($"{key}:{secret}")); client.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Basic", auth); var content = new System.Net.Http.FormUrlEncodedContent(new[] { new KeyValuePair("test[success]", _testPassed ? "1" : "0") }); client.PutAsync($"https://api.testingbot.com/v1/tests/{sessionId}", content).Wait(); } catch (Exception) { // Ignore reporting errors so they don't fail an otherwise passing test } finally { Driver.Quit(); } } } } // Usage in test class public class MyTests : TestBase { [Fact] public void TestSomething() { try { Driver.Navigate().GoToUrl("https://example.com"); Assert.Equal("Example Domain", Driver.Title); } catch { MarkTestFailed(); throw; } } } ``` ## Other C# Framework examples - [NUnit](https://testingbot.com/support/web-automate/selenium/csharp/nunit) An unit testing framework that is open source written in C#. - [PNunit](https://testingbot.com/support/web-automate/selenium/csharp/pnunit) With PNUnit you can run several tests in parallel. - [SpecFlow](https://testingbot.com/support/web-automate/selenium/csharp/specflow) SpecFlow allows you to run Automated .NET tests using Cucumber-compatible Gherkin syntax. - [MSTest](https://testingbot.com/support/web-automate/selenium/csharp/mstest) MSTest framework is a test framework which is included, by default, with Microsoft Visual Studio. - [Reqnroll](https://testingbot.com/support/web-automate/selenium/csharp/reqnroll) Reqnroll is the successor to SpecFlow for BDD testing with Gherkin syntax. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/csharp/specflow **SpecFlow reached End of Life (EOL) on December 31, 2024.** For new projects, we recommend using [Reqnroll](https://testingbot.com/support/web-automate/selenium/csharp/reqnroll), the community-maintained successor to SpecFlow. Reqnroll is fully compatible with SpecFlow and provides an easy migration path. # Your first C# test with SpecFlow See our [SpecFlow example repository](https://github.com/testingbot/specflow-example) for a simple example on how to run SpecFlow tests in parallel on TestingBot. SpecFlow is an open-source .NET utility which allows you to write tests using Cucumber-compatible Gherkin syntax. ## Installation You can install SpecFlow via the [NuGet Gallery](https://www.nuget.org/) - Create a new project in Visual Studio. - In the Visual Studio Tools menu, go to **Library Package Manager \> Manage Nuget Package for Solution**. - This will open the Manage NuGet Packages dialog. Click **Online** , then **Next**. - In the **Search Packages** field, enter **SpecFlow** and click **Search**. - Select **SpecFlow** from the search results and click **Install**. For more details, please visit [SpecFlow installation Document](https://specflow.org/getting-started/#InstallSetup). ### Sample Test This is a sample test case in SpecFlow: ```gherkin Feature: Google Scenario Outline: Can find search results Given I am on the google page for and When I search for "TestingBot" Then I should see title "TestingBot - Google Search" Examples: | profile | environment | | single | chrome | ``` ```csharp [Binding] public class SingleSteps { private readonly ScenarioContext _scenarioContext; private IWebDriver _driver; private TestingBotDriver _tbDriver; public SingleSteps(ScenarioContext scenarioContext) { _scenarioContext = scenarioContext; _tbDriver = (TestingBotDriver)_scenarioContext["tbDriver"]; } [Given(@"I am on the google page for (.*) and (.*)")] public void GivenIAmOnTheGooglePage(string profile, string environment) { _driver = _tbDriver.Init(profile, environment); _driver.Navigate().GoToUrl("https://www.google.com/ncr"); } [When(@"I search for ""(.*)""")] public void WhenISearchFor(string keyword) { var q = _driver.FindElement(By.Name("q")); q.SendKeys(keyword); q.Submit(); } [Then(@"I should see title ""(.*)""")] public void ThenIShouldSeeTitle(string title) { Thread.Sleep(5000); Assert.That(_driver.Title, Is.EqualTo(title)); } } ``` Now you need to create the **TestingBotDriver** we will be using. The hooks class accepts `ScenarioContext` via constructor injection, `ScenarioContext.Current` is deprecated and not safe under parallel execution. ```csharp [Binding] public sealed class TestingBotHooks { private readonly ScenarioContext _scenarioContext; private TestingBotDriver _tbDriver; public TestingBotHooks(ScenarioContext scenarioContext) { _scenarioContext = scenarioContext; } [BeforeScenario] public void BeforeScenario() { _tbDriver = new TestingBotDriver(_scenarioContext); _scenarioContext["tbDriver"] = _tbDriver; } [AfterScenario] public void AfterScenario() { _tbDriver.Cleanup(); } } ``` ```csharp public class TestingBotDriver { private IWebDriver driver; private ScenarioContext context; public TestingBotDriver(ScenarioContext context) { this.context = context; } public IWebDriver Init(string profile, string environment) { NameValueCollection caps = ConfigurationManager.GetSection("capabilities/" + profile) as NameValueCollection; NameValueCollection settings = ConfigurationManager.GetSection("environments/" + environment) as NameValueCollection; // Build the right options class for the requested browser. ChromeOptions hardcodes // browserName=chrome, so we can't just always use it. string browserName = settings["browserName"] ?? "chrome"; DriverOptions options = browserName.ToLowerInvariant() switch { "firefox" => new FirefoxOptions(), "microsoftedge" => new EdgeOptions(), "safari" => new SafariOptions(), _ => new ChromeOptions() }; options.BrowserVersion = settings["browserVersion"]; options.PlatformName = settings["platformName"]; // Standard W3C capabilities go on the typed setters above; everything else // (vendor-specific TestingBot caps) goes inside tb:options. var tbOptions = new Dictionary(); foreach (string key in caps.AllKeys) { tbOptions[key] = caps[key]; } string tbKey = Environment.GetEnvironmentVariable("TESTINGBOT_KEY") ?? ConfigurationManager.AppSettings.Get("key"); string tbSecret = Environment.GetEnvironmentVariable("TESTINGBOT_SECRET") ?? ConfigurationManager.AppSettings.Get("secret"); tbOptions["key"] = tbKey; tbOptions["secret"] = tbSecret; options.AddAdditionalOption("tb:options", tbOptions); driver = new RemoteWebDriver(new Uri("https://" + ConfigurationManager.AppSettings.Get("server") + "/wd/hub"), options); return driver; } public void Cleanup() { driver.Quit(); } } ``` Now we need to create the **config file** which the code above uses to determine various settings. ```xml
``` ## Specify Browsers & Devices To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field. ```xml ``` To see how to do this, please select a combination of browser, version and platform in the drop-down menus below. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites We've built [TestingBot Tunnel](https://testingbot.com/support/tunnel), to provide you with a secure way to run tests against your staged/internal webapps. Please see our [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for more information about this easy to use tunneling solution. The example below shows how to easily run a SpecFlow test with our Tunnel: 1. [Download our tunnel](https://testingbot.com/support/tunnel) and start the tunnel: ```bash java -jar testingbot-tunnel.jar key secret ``` 2. Adjust your test: instead of pointing to `'hub.testingbot.com/wd/hub'` like the example above - change it to point to your tunnel's IP address. Assuming you run the tunnel on the same machine you run your tests, change to `'localhost:4445/wd/hub'`. localhost is the machine running the tunnel, 4445 is the default port of the tunnel. This way your test will go securely through the tunnel to TestingBot and back: ```xml ``` ## Run tests in Parallel Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time. You can run the same tests on all different browser configurations or run different tests all on the same browser configuration. TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time. To run multiple tests at the same time with SpecFlow, modify the **config file** like this: ```xml
``` ```csharp // Add to AssemblyInfo.cs [assembly: Parallelizable(ParallelScope.Fixtures)] ``` The test cases you want to run in parallel need to be written like this: ```gherkin // Google Feature for parallel run Feature: Google Scenario Outline: Can find search results Given I am on the google page for and When I search for "TestingBot" Then I should see title "TestingBot - Google Search" Examples: | profile | environment | | parallel | chrome | | parallel | firefox | | parallel | safari | | parallel | edge | ``` You can now run the tests in parallel by running the tests with fixture **parallel** from **Test Explorer**. ### Queuing Every plan we provide comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available. ## Mark tests as passed/failed To see if a test passed or not in our member area, or to send additional meta-data to TestingBot, you can use our API. Please see the example below on how to notify TestingBot about the test success state: ```csharp [TearDown] public void CleanUp() { bool passed = TestContext.CurrentContext.Result.Status == TestStatus.Passed; try { // Logs the result to TestingBot ((IJavaScriptExecutor)driver).ExecuteScript("tb:test-result=" + (passed ? "passed" : "failed")); } finally { // Terminates the remote webdriver session driver.Quit(); } } ``` ## Other C# Framework examples - [Reqnroll](https://testingbot.com/support/web-automate/selenium/csharp/reqnroll) Reqnroll is the community-maintained successor to SpecFlow for BDD testing with Gherkin syntax. **Recommended for new projects.** - [NUnit](https://testingbot.com/support/web-automate/selenium/csharp/nunit) An unit testing framework that is open source written in C#. - [PNunit](https://testingbot.com/support/web-automate/selenium/csharp/pnunit) With PNUnit you can run several tests in parallel. - [MSTest](https://testingbot.com/support/web-automate/selenium/csharp/mstest) MSTest framework is a test framework which is included, by default, with Microsoft Visual Studio. - [xUnit](https://testingbot.com/support/web-automate/selenium/csharp/xunit) xUnit.net is a modern, extensible testing framework for .NET, widely used in the .NET ecosystem. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/csharp/reqnroll # C# BDD Testing with Reqnroll [Reqnroll](https://reqnroll.net/) is an open-source .NET test automation framework for Behavior-Driven Development (BDD). It is the community-maintained successor to SpecFlow and is fully compatible with Gherkin syntax. Reqnroll works with all major .NET versions (.NET 6.0, 7.0, 8.0, 9.0) and integrates with popular test frameworks: - NUnit - xUnit - MSTest - TUnit **Migrating from SpecFlow?** Reqnroll is based on the SpecFlow codebase and provides an easy migration path. See the [migration section](https://testingbot.com#migration) below. ## Installation First, install the **Reqnroll for Visual Studio 2022** extension from the Visual Studio Marketplace to get syntax highlighting, navigation and project templates. ### Create a new project Create a new test project and add the required packages: ```bash # Create a new NUnit test project dotnet new nunit -n ReqnrollTests cd ReqnrollTests # Add Reqnroll with NUnit integration dotnet add package Reqnroll.NUnit # Add Selenium WebDriver packages dotnet add package Selenium.WebDriver dotnet add package Selenium.Support ``` Or add these packages to your `.csproj` file: ```xml ``` Alternative packages for other test frameworks: - `Reqnroll.xUnit` - for xUnit 2.x - `Reqnroll.xUnit.v3` - for xUnit 3.x - `Reqnroll.MsTest` - for MSTest - `Reqnroll.TUnit` - for TUnit ## Your first Reqnroll test Reqnroll tests consist of three parts: a Feature file (Gherkin), Step Definitions (C#), and a Driver helper class. ### 1. Feature File Create a `Features` folder and add `Search.feature`: ```yaml Feature: Google Search Scenario: Search for TestingBot Given I am on the Google homepage When I search for "TestingBot" Then the page title should contain "TestingBot" Scenario Outline: Search on multiple browsers Given I am using on And I am on the Google homepage When I search for "TestingBot" Then the page title should contain "TestingBot" Examples: | browser | platform | | chrome | WIN10 | | firefox | WIN10 | | safari | TAHOE | ``` ### 2. Driver Helper Class Create a helper class to manage the WebDriver connection to TestingBot: ```csharp using OpenQA.Selenium; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Firefox; using OpenQA.Selenium.Safari; using OpenQA.Selenium.Remote; namespace ReqnrollTests.Support; public class TestingBotDriver : IDisposable { public IWebDriver Driver { get; private set; } private bool _testPassed = true; public void Initialize(string browser = "chrome", string version = "latest", string platform = "WIN10") { DriverOptions options = browser.ToLower() switch { "firefox" => new FirefoxOptions(), "safari" => new SafariOptions(), _ => new ChromeOptions() }; options.BrowserVersion = version; options.PlatformName = platform; options.AddAdditionalOption("tb:options", new Dictionary { ["key"] = Environment.GetEnvironmentVariable("TB_KEY") ?? "", ["secret"] = Environment.GetEnvironmentVariable("TB_SECRET") ?? "", ["name"] = "Reqnroll Test" }); Driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), options ); } public void MarkTestFailed() => _testPassed = false; public void Dispose() { if (Driver != null) { var status = _testPassed ? "passed" : "failed"; ((IJavaScriptExecutor)Driver).ExecuteScript($"tb:test-result={status}"); Driver.Dispose(); } } } ``` ### 3. Step Definitions Create `StepDefinitions/SearchSteps.cs`: ```csharp using OpenQA.Selenium; using Reqnroll; using ReqnrollTests.Support; namespace ReqnrollTests.StepDefinitions; [Binding] public class SearchSteps { private readonly TestingBotDriver _tbDriver; public SearchSteps(TestingBotDriver tbDriver) { _tbDriver = tbDriver; } [Given("I am on the Google homepage")] public void GivenIAmOnTheGoogleHomepage() { if (_tbDriver.Driver == null) _tbDriver.Initialize(); _tbDriver.Driver.Navigate().GoToUrl("https://www.google.com"); } [Given("I am using (.*) on (.*)")] public void GivenIAmUsingBrowserOnPlatform(string browser, string platform) { _tbDriver.Initialize(browser, "latest", platform); } [When("I search for {string}")] public void WhenISearchFor(string searchTerm) { var searchBox = _tbDriver.Driver.FindElement(By.Name("q")); searchBox.SendKeys(searchTerm); searchBox.Submit(); } [Then("the page title should contain {string}")] public void ThenThePageTitleShouldContain(string expectedText) { // Wait for results to load Thread.Sleep(2000); Assert.That(_tbDriver.Driver.Title, Does.Contain(expectedText)); } } ``` ### 4. Hooks for Setup/Teardown Create `Hooks/TestHooks.cs` to manage driver lifecycle: ```csharp using Reqnroll; using ReqnrollTests.Support; namespace ReqnrollTests.Hooks; [Binding] public class TestHooks { private readonly TestingBotDriver _tbDriver; public TestHooks(TestingBotDriver tbDriver) { _tbDriver = tbDriver; } [AfterScenario] public void AfterScenario(ScenarioContext scenarioContext) { if (scenarioContext.TestError != null) { _tbDriver.MarkTestFailed(); } _tbDriver.Dispose(); } } ``` Run your tests with: ```bash TB_KEY=your_key TB_SECRET=your_secret dotnet test ``` ## Specify Browsers & Devices Configure browsers and platforms using Selenium 4's browser-specific options classes: - `ChromeOptions` for Chrome - `FirefoxOptions` for Firefox - `EdgeOptions` for Microsoft Edge - `SafariOptions` for Safari Example using ChromeOptions: ```csharp var options = new ChromeOptions(); options.BrowserVersion = "latest"; options.PlatformName = "WIN10"; options.AddAdditionalOption("tb:options", new Dictionary { ["key"] = "YOUR_TB_KEY", ["secret"] = "YOUR_TB_SECRET", ["name"] = "My Reqnroll Test" }); var driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), options ); ``` To see how to configure different browsers and platforms, use the picker below: ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Testing Internal Websites Use [TestingBot Tunnel](https://testingbot.com/support/tunnel) to securely test internal or staged websites. **1.** [Download the tunnel](https://testingbot.com/support/tunnel) and start it: ```bash java -jar testingbot-tunnel.jar key secret ``` **2.** Update your driver initialization to connect through the tunnel: ```csharp // Connect through the tunnel instead of directly to TestingBot Driver = new RemoteWebDriver( new Uri("http://localhost:4445/wd/hub"), options ); // Now you can access internal URLs in your tests Driver.Navigate().GoToUrl("http://internal.yourcompany.com"); ``` ## Run tests in Parallel Reqnroll supports parallel test execution when using NUnit or xUnit as the test runner. ### NUnit Parallel Execution Add to your `AssemblyInfo.cs` or any file: ```csharp using NUnit.Framework; [assembly: Parallelizable(ParallelScope.Fixtures)] [assembly: LevelOfParallelism(5)] ``` Configure in `reqnroll.json`: ```json { "$schema": "https://schemas.reqnroll.net/reqnroll-config-latest.json", "bindingCulture": { "name": "en-US" } } ``` ### xUnit Parallel Execution Add `xunit.runner.json`: ```json { "parallelizeAssembly": true, "parallelizeTestCollections": true, "maxParallelThreads": 5 } ``` ### Queuing Every TestingBot plan has a limit on parallel tests. If you exceed this limit, additional tests are queued (up to 6 minutes) and run as slots become available. ## Mark tests as passed/failed Report test results to TestingBot using JavaScript execution in your `[AfterScenario]` hook: ```csharp using OpenQA.Selenium; using Reqnroll; [Binding] public class TestHooks { private readonly TestingBotDriver _tbDriver; public TestHooks(TestingBotDriver tbDriver) { _tbDriver = tbDriver; } [AfterScenario] public void AfterScenario(ScenarioContext scenarioContext) { if (_tbDriver.Driver != null) { var status = scenarioContext.TestError == null ? "passed" : "failed"; ((IJavaScriptExecutor)_tbDriver.Driver).ExecuteScript($"tb:test-result={status}"); _tbDriver.Driver.Dispose(); } } } ``` ## Migrating from SpecFlow Reqnroll is designed as a drop-in replacement for SpecFlow. Most projects can migrate with minimal changes: ### 1. Update NuGet Packages ```bash # Remove SpecFlow packages dotnet remove package SpecFlow dotnet remove package SpecFlow.NUnit # Add Reqnroll packages dotnet add package Reqnroll.NUnit ``` ### 2. Update Namespaces Replace `TechTalk.SpecFlow` with `Reqnroll` in your code: ```csharp // Before (SpecFlow) using TechTalk.SpecFlow; // After (Reqnroll) using Reqnroll; ``` ### 3. Update Configuration Rename `specflow.json` to `reqnroll.json` and update the schema reference: ```json { "$schema": "https://schemas.reqnroll.net/reqnroll-config-latest.json" } ``` ### 4. Update Visual Studio Extension Install the "Reqnroll for Visual Studio 2022" extension and uninstall the SpecFlow extension. For a complete migration guide, see the [official SpecFlow to Reqnroll migration guide](https://reqnroll.net/news/2024/02/from-specflow-to-reqnroll-why-and-how/). ## Other C# Framework examples - [NUnit](https://testingbot.com/support/web-automate/selenium/csharp/nunit) An unit testing framework that is open source written in C#. - [xUnit](https://testingbot.com/support/web-automate/selenium/csharp/xunit) xUnit.net is a modern, extensible testing framework for .NET, widely used in the .NET ecosystem. - [MSTest](https://testingbot.com/support/web-automate/selenium/csharp/mstest) MSTest framework is a test framework which is included, by default, with Microsoft Visual Studio. - [SpecFlow](https://testingbot.com/support/web-automate/selenium/csharp/specflow) SpecFlow (deprecated) - the predecessor to Reqnroll for BDD testing with Gherkin syntax. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/selenium4 # Selenium 4 This page will help you in upgrading your existing Selenium WebDriver tests to use Selenium 4, with the [WebDriver W3C Protocol](https://www.w3.org/TR/webdriver1/). TestingBot has full support for this new Selenium version. Below is more information on how to make this transition. ## Updating your tests With the new W3C protocol, some restrictions apply to using capabilities with your test. Before the W3C protocol (the JSONWP protocol), you could simply add [all our custom TestingBot capabilities](https://testingbot.com/support/web-automate/selenium/test-options) to your test in the `desiredCapabilities`. With the new W3C protocol, only some W3C WebDriver capabilities are allowed inside the `capabilities`: - `browserName` - `browserVersion` - `platformName` - `acceptInsecureCerts` - `pageLoadStrategy` - `proxy` - `timeouts` - `unhandledPromptBehavior` All other capabilities are treated as [Extension capabilities](https://www.w3.org/TR/webdriver1/#dfn-extension-capability) and should be namespaced (include a vendor prefix). For [TestingBot specific capabilities](https://testingbot.com/support/web-automate/selenium/test-options), you'll need to wrap all capabilities inside a `tb:options` namespace. Please see the table below on how the capabilities need to be changed: | Capability JSONWP | Capability W3C | | --- | --- | | `browserName` | `browserName` | | `version` | `browserVersion` | | `platform` | `platformName` | | `selenium-version`, `chromedriverVersion`, `extra`, `name` [and more](https://testingbot.com/support/web-automate/selenium/test-options) | ```bash "tb:options" : { "selenium-version": '...', "chromedriverVersion": '...', "extra": '...', "name": '...' } ``` | ## Example [Legacy](https://testingbot.com#) [Selenium 4](https://testingbot.com#) ### Ruby ```ruby #!/usr/bin/env ruby require 'rubygems' require 'selenium-webdriver' caps = { :browserName => "chrome", :version => "latest-1", :platform => "WIN10", :screenrecorder => true, :build => "testbuild", :name => "testname" } client = Selenium::WebDriver::Remote::Http::Default.new client.timeout = 480 driver = Selenium::WebDriver.for( :remote, :url => "https://API_KEY:API_SECRET@hub.testingbot.com/wd/hub", :http_client => client, :desired_capabilities => caps) driver.navigate.to "https://www.google.com" element = driver.find_element(:name, 'q') element.send_keys "TestingBot" element.submit puts driver.title driver.quit ``` ### PHP ```php "chrome", 'version' => "latest-1", 'platform' => "WIN10", 'screenrecorder' => true, 'build' => "testbuild", 'name' => "testname" ); $web_driver = RemoteWebDriver::create( "https://api_key:api_secret@hub.testingbot.com/wd/hub", $capabilities, 240000 ); $web_driver->get("https://google.com"); $element = $web_driver->findElement(WebDriverBy::name("q")); if($element) { $element->sendKeys("TestingBot"); $element->submit(); } print $web_driver->getTitle(); $web_driver->quit(); ?> ``` ### Java ```java import org.openqa.selenium.By; import org.openqa.selenium.Platform; import org.openqa.selenium.WebDriver; import org.openqa.selenium.WebElement; import org.openqa.selenium.remote.DesiredCapabilities; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URL; public class JavaSample { public static final String KEY = "KEY"; public static final String SECRET = "SECRET"; public static final String HUB_URL = "https://" + KEY + ":" + SECRET + "@hub.testingbot.com/wd/hub"; public static void main(String[] args) throws Exception { DesiredCapabilities caps = new DesiredCapabilities(); caps.setCapability("browserName", "chrome"); caps.setCapability("version", "latest-1"); caps.setCapability("platform", "WIN10"); caps.setCapability("screenrecorder", true); caps.setCapability("build", "testbuild"); caps.setCapability("name", "testname"); WebDriver driver = new RemoteWebDriver(new URL(HUB_URL), caps); driver.get("https://www.google.com/ncr"); WebElement element = driver.findElement(By.name("q")); element.sendKeys("TestingBot"); element.submit(); System.out.println(driver.getTitle()); driver.quit(); } } ``` ### Python ```python import unittest import sys from selenium import webdriver from selenium.webdriver.common.keys import Keys from selenium.webdriver.common.by import By from selenium.webdriver.common.desired_capabilities import DesiredCapabilities from testingbotclient import TestingBotClient class TestTestingBotClient(unittest.TestCase): def setUp(self): desired_cap = { 'browserName': 'chrome', 'version': 'latest-1', 'platform': 'WIN10', 'screenrecorder': True, 'build': 'testbuild', 'name' : 'testname', } self.driver = webdriver.Remote( command_executor='https://key:secret@hub.testingbot.com/wd/hub', desired_capabilities=desired_cap) def test_google_example(self): self.driver.get("https://www.google.com") if not "Google" in self.driver.title: raise Exception("Unable to load google page!") elem = self.driver.find_element(By.NAME, "q") elem.send_keys("TestingBot") elem.submit() def tearDown(self): passed = sys.exc_info() == (None, None, None) session_id = self.driver.session_id try: tb_client = TestingBotClient('key', 'secret') tb_client.tests.update_test(session_id, name=self._testMethodName, passed=passed) finally: self.driver.quit() if __name__ == ' __main__': unittest.main() ``` ### JavaScript ```javascript var wd = require('wd'), testingbotKey = "api_key", testingbotSecret = "api_secret" desiredCaps = { 'browserName': 'chrome', 'version': 'latest-1', 'platform': 'WIN10', 'screenrecorder': true, 'build': 'testbuild', 'name' : 'testname', } driver = wd.remote("https://" + testingbotKey + ":" + testingbotSecret + "@" + "hub.testingbot.com/wd/hub") driver.init(desiredCaps, function() { driver.get('https://www.google.com', function() { driver.title(function(err, title) { console.log(title) driver.quit() }) }) }) ``` ### Ruby ```ruby #!/usr/bin/env ruby require 'rubygems' require 'selenium-webdriver' options = Selenium::WebDriver::Chrome::Options.new options.browser_version = 'latest-1' options.platform_name = 'WIN10' options.add_option('tb:options', { 'screenrecorder' => true, 'build' => 'testbuild', 'name' => 'testname' }) client = Selenium::WebDriver::Remote::Http::Default.new client.timeout = 480 driver = Selenium::WebDriver.for( :remote, url: "https://API_KEY:API_SECRET@hub.testingbot.com/wd/hub", http_client: client, options: options) driver.navigate.to "https://www.google.com" element = driver.find_element(:name, 'q') element.send_keys "TestingBot" element.submit puts driver.title driver.quit ``` ### PHP ```php "chrome", 'browserVersion' => "latest-1", 'platformName' => "WIN10", 'tb:options' => array( 'screenrecorder' => true, 'build' => "testbuild", 'name' => "testname" ) ); $web_driver = RemoteWebDriver::create( "https://api_key:api_secret@hub.testingbot.com/wd/hub", $capabilities, 240000 ); $web_driver->get("https://google.com"); $element = $web_driver->findElement(WebDriverBy::name("q")); if($element) { $element->sendKeys("TestingBot"); $element->submit(); } print $web_driver->getTitle(); $web_driver->quit(); ?> ``` ### Java ```java import org.openqa.selenium.By; import org.openqa.selenium.WebDriver; import org.openqa.selenium.WebElement; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URL; import java.util.HashMap; import java.util.Map; public class JavaSample { public static final String KEY = "KEY"; public static final String SECRET = "SECRET"; public static final String HUB_URL = "https://" + KEY + ":" + SECRET + "@hub.testingbot.com/wd/hub"; public static void main(String[] args) throws Exception { Map tbOptions = new HashMap<>(); tbOptions.put("screenrecorder", true); tbOptions.put("build", "testbuild"); tbOptions.put("name", "testname"); ChromeOptions options = new ChromeOptions(); options.setBrowserVersion("latest-1"); options.setPlatformName("WIN10"); options.setCapability("tb:options", tbOptions); WebDriver driver = new RemoteWebDriver(new URL(HUB_URL), options); driver.get("https://www.google.com/ncr"); WebElement element = driver.findElement(By.name("q")); element.sendKeys("TestingBot"); element.submit(); System.out.println(driver.getTitle()); driver.quit(); } } ``` ### Python ```python import unittest import sys from selenium import webdriver from selenium.webdriver.common.keys import Keys from selenium.webdriver.common.by import By from selenium.webdriver.chrome.options import Options from testingbotclient import TestingBotClient class TestTestingBotClient(unittest.TestCase): def setUp(self): options = Options() options.browser_version = 'latest-1' options.platform_name = 'WIN10' options.set_capability('tb:options', { 'screenrecorder': True, 'build': 'testbuild', 'name': 'testname' }) self.driver = webdriver.Remote( command_executor='https://key:secret@hub.testingbot.com/wd/hub', options=options) def test_google_example(self): self.driver.get("https://www.google.com") if not "Google" in self.driver.title: raise Exception("Unable to load google page!") elem = self.driver.find_element(By.NAME, "q") elem.send_keys("TestingBot") elem.submit() def tearDown(self): passed = sys.exc_info() == (None, None, None) session_id = self.driver.session_id try: tb_client = TestingBotClient('key', 'secret') tb_client.tests.update_test(session_id, name=self._testMethodName, passed=passed) finally: self.driver.quit() if __name__ == ' __main__': unittest.main() ``` ### JavaScript ```javascript const { Builder, By } = require('selenium-webdriver'); const testingbotKey = "api_key"; const testingbotSecret = "api_secret"; const capabilities = { 'browserName': 'chrome', 'browserVersion': 'latest-1', 'platformName': 'WIN10', 'tb:options': { 'screenrecorder': true, 'build': 'testbuild', 'name': 'testname' } }; (async function example() { const driver = await new Builder() .usingServer(`https://${testingbotKey}:${testingbotSecret}@hub.testingbot.com/wd/hub`) .withCapabilities(capabilities) .build(); try { await driver.get('https://www.google.com'); console.log(await driver.getTitle()); } finally { await driver.quit(); } })(); ``` ## New Selenium 4 Features Below are some of the new features introduced by Selenium 4 which you can use in your TestingBot tests. ### New Window Create or switch to a new (blank) tab or window. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby @driver.switch_to.new_window(:window) @driver.manage.window.position = Selenium::WebDriver::Point.new(200, 400) @driver.switch_to.new_window(:tab) ``` ```java driver.switchTo().newWindow(WindowType.WINDOW); driver.manage().window().setPosition(new Point(200, 400)); driver.switchTo().newWindow(WindowType.TAB); ``` ```python driver.switch_to.new_window('window') driver.set_window_position(200, 400) driver.switch_to.new_window('tab') ``` ```csharp Driver.SwitchTo().NewWindow(WindowType.Window); Driver.Manage().Window.Position = new Point(200, 400); Driver.SwitchTo().NewWindow(WindowType.Tab); ``` ### Print Page Print the current page as a `PDF`, available in Chrome, Edge and Firefox. You can pass additional options to customize the `PDF`, such as `page size`, `range`, `margins`, `background` and shrink to fit. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby driver.navigate.to url_for("https://testingbot.com") path = "PrintPage.pdf" driver.save_print_page path ``` ```java Path printPage = Paths.get(directory + "PrintPage.pdf"); Pdf print = driver.print(new PrintOptions()); Files.write(printPage, OutputType.BYTES.convertFromBase64Png(print.getContent())); ``` ```python from base64 import b64decode driver.get("https://testingbot.com") pdf = b64decode(driver.print_page()) with open("print_page.pdf", 'wb') as f: f.write(pdf) ``` ```csharp Driver.Navigate().GoToUrl("https://testingbot.com"); var parentFullName = Directory.GetParent(Environment.CurrentDirectory)?.Parent?.Parent?.FullName; ((ISupportsPrint) Driver).Print(new PrintOptions()).SaveAsFile(parentFullName + "/PrintPage.pdf"); ``` For Chrome and Edge, you need to use `headless` mode to print pages. ### Relative Locators With Relative Locators, you can identify DOM elements in relationship to other DOM elements. You can use a more natural syntax such as: - above - below - left of - right of - near [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby @driver.get("https://testingbot.com") element = @driver.find_element(relative: {tag_name: 'div', left: {id: 'outer-orbit'}, below: {id: 'circle-orbit-container'}}) ``` ```java driver.get("https://testingbot.com"); WebElement element = driver.findElement(with(By.tagName("div")) .toLeftOf(By.id("outer-orbit")) .below(By.id("circle-orbit-container"))); ``` ```python from selenium.webdriver.common.by import By from selenium.webdriver.support.relative_locator import locate_with driver.get('https://testingbot.com') anchor = driver.find_element(By.ID, "outer-orbit") below_of = driver.find_element(By.ID, "circle-orbit-container") element = driver.find_element(locate_with(By.TAG_NAME, "div") .to_left_of(anchor) .below(below_of)) ``` ```csharp Driver.Navigate().GoToUrl("https://testingbot.com"); IWebElement element = Driver.FindElement(RelativeBy.WithLocator(By.TagName("div")) .LeftOf(By.Id("outer-orbit")) .Below(By.Id("circle-orbit-container"))); ``` ### Retrieve Timeouts Selenium 4 now allows you to query the driver for the current timeout values. Please see the examples below on how to do this, for example with implicit waits, pageload timeouts or script timeout. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby @driver.manage.timeouts.implicit_wait = 1 p @driver.manage.timeouts.implicit_wait p @driver.manage.timeouts.page_load p @driver.manage.timeouts.script ``` ```java WebDriver.Timeouts timeouts = driver.manage().timeouts(); timeouts.pageLoadTimeout(Duration.ofSeconds(10)); timeouts.implicitlyWait(Duration.ofMillis(11)); timeouts.scriptTimeout(Duration.ofSeconds(12)); timeouts.getPageLoadTimeout(); Assertions.assertEquals(Duration.ofSeconds(10), timeouts.getPageLoadTimeout()); Assertions.assertEquals(Duration.ofMillis(11), timeouts.getImplicitWaitTimeout()); Assertions.assertEquals(Duration.ofSeconds(12), timeouts.getScriptTimeout()); ``` ```python from selenium.webdriver.common.timeouts import Timeouts timeouts = Timeouts() timeouts.implicit_wait = 1 driver.timeouts = timeouts assert driver.timeouts.implicit_wait == 1 assert driver.timeouts.page_load == 20 assert driver.timeouts.script == 20 ``` ```csharp Driver.Manage().Timeouts().ImplicitWait = TimeSpan.FromSeconds(0.4); Assert.AreEqual(TimeSpan.FromSeconds(0.4), Driver.Manage().Timeouts().ImplicitWait); Assert.AreEqual(TimeSpan.FromSeconds(4), Driver.Manage().Timeouts().PageLoad); Assert.AreEqual(TimeSpan.FromSeconds(40), Driver.Manage().Timeouts().AsynchronousJavaScript); ``` ### Full Page Screenshots Firefox Only Firefox implemented a feature to take screenshots of the entire page, not just the viewport, which is the default setting for other browsers or for Selenium 3. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby @driver.get('https://testingbot.com') @driver.save_screenshot('FullScreenshotOnFirefox.png', full_page: true) ``` ```java WebDriver augmentedDriver = new Augmenter().augment(driver); File file = ((HasFullPageScreenshot) augmentedDriver).getFullPageScreenshotAs(OutputType.FILE); Path fullPageScreenshot = Paths.get(directory + "FullScreenshotOnFirefox.png"); Files.move(file.toPath(), fullPageScreenshot); ``` ```python driver.get('https://testingbot.com') driver.get_full_page_screenshot_as_file('FullScreenshotOnFirefox.png') ``` ```csharp IHasCommandExecutor hasCommandExecutor = Driver as IHasCommandExecutor; var addFullPageScreenshotCommandInfo = new HttpCommandInfo(HttpCommandInfo.GetCommand, "/session/{sessionId}/moz/screenshot/full"); hasCommandExecutor.CommandExecutor.TryAddCommand("fullPageScreenshot", addFullPageScreenshotCommandInfo); SessionId sessionId = ((RemoteWebDriver)Driver).SessionId; var fullPageScreenshotCommand = new Command(sessionId, "fullPageScreenshot", null); Driver.Navigate().GoToUrl("https://testingbot.com"); var screenshotResponse = hasCommandExecutor.CommandExecutor.Execute(fullPageScreenshotCommand); string base64 = screenshotResponse.Value.ToString(); Screenshot image = new Screenshot(base64); var parentFullName = Directory.GetParent(Environment.CurrentDirectory)?.Parent?.Parent?.FullName; image.SaveAsFile(parentFullName + "/FullScreenshotOnFirefox.png", ScreenshotImageFormat.Png); ``` ### Network Conditions Chrome and Edge only Selenium 4 offers the possibility to modify network conditions for your tests, such as: - Setting latency - Modifying upstream/downstream throughput - Going into offline mode [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby @driver.network_conditions = {offline: true, latency: 1, throughput: 10000} ``` ```java WebDriver augmentedDriver = new Augmenter().augment(driver); ChromiumNetworkConditions networkConditions = new ChromiumNetworkConditions(); networkConditions.setOffline(true); ((HasNetworkConditions) augmentedDriver).setNetworkConditions(networkConditions); try { driver.get("https://testingbot.com"); } catch (WebDriverException ex) { ((HasNetworkConditions) augmentedDriver).setNetworkConditions(new ChromiumNetworkConditions()); } ``` ```python driver.set_network_conditions( offline=True, latency=1, throughput=10000 ) ``` ```csharp INetworkConditions networkConditions = new NetworkConditions() { IsOffline = true, Latency = TimeSpan.FromMilliseconds(1), DownloadThroughput = 10000, UploadThroughput = 10000 }; ((IChromiumNetworkConditions)driver).NetworkConditions = networkConditions; ``` TestingBot has been offering [throttling, mocking and intercepting](https://testingbot.com/support/web-automate/selenium/network) network requests before Selenium 4. ### Change Preferences During Session Firefox only Before Selenium 4, it was only possible to set preferences (through capabilities) at the start of a test. Firefox now provides the option to change preferences during a session. You can do this by toggling the context between `chrome` and `content`. Make sure to switch back to the correct context, or some commands might fail. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby @driver.get("https://www.google.com") language = @driver.find_element(id: "gws-output-pages-elements-homepage_additional_languages__als") p language @driver.context = 'chrome' @driver.execute_script("Services.prefs.setStringPref('intl.accept_languages', 'es-ES')") @driver.context = 'content' @driver.navigate.refresh language = @driver.find_element(id: "gws-output-pages-elements-homepage_additional_languages__als") p language @driver.quit ``` ```java driver.get("https://www.google.com"); String lang1 = driver.findElement(By.id("gws-output-pages-elements-homepage_additional_languages__als")).getText(); Assertions.assertTrue(lang1.contains("offered in")); WebDriver augmentedDriver = new Augmenter().augment(driver); ((HasContext) augmentedDriver).setContext(FirefoxCommandContext.CHROME); ((JavascriptExecutor) driver).executeScript("Services.prefs.setStringPref('intl.accept_languages', 'es-ES')"); ((HasContext) augmentedDriver).setContext(FirefoxCommandContext.CONTENT); driver.navigate().refresh(); String lang2 = driver.findElement(By.id("gws-output-pages-elements-homepage_additional_languages__als")).getText(); Assertions.assertTrue(lang2.contains("Ofrecido por")); ``` This feature is not yet supported in the Python bindings. ```csharp IHasCommandExecutor hasCommandExecutor = Driver as IHasCommandExecutor; var getContextCommandInfo = new HttpCommandInfo(HttpCommandInfo.GetCommand, "/session/{sessionId}/moz/context"); var setContextCommandInfo = new HttpCommandInfo(HttpCommandInfo.PostCommand, "/session/{sessionId}/moz/context"); hasCommandExecutor.CommandExecutor.TryAddCommand("getContext", getContextCommandInfo); hasCommandExecutor.CommandExecutor.TryAddCommand("setContext", setContextCommandInfo); SessionId sessionId = ((RemoteWebDriver)Driver).SessionId; Driver.Navigate().GoToUrl("https://www.google.com"); var element = Driver.FindElement(By.CssSelector("#gws-output-pages-elements-homepage_additional_languages__als")); Assert.IsTrue(element.Text.Contains("offered in")); try { Driver.ExecuteJavaScript("Services.prefs.setStringPref('intl.accept_languages', 'es-ES')"); Assert.Fail("Can not change Service prefs in content context, so previous method should fail"); } catch (WebDriverException) { // This is expected } var getContextCommand = new Command(sessionId, "getContext", null); Response response = hasCommandExecutor.CommandExecutor.Execute(getContextCommand); Assert.AreEqual("content", response.Value); Dictionary payload = new Dictionary(); payload.Add("context", "chrome"); var setContextCommand = new Command(sessionId, "setContext", payload); hasCommandExecutor.CommandExecutor.Execute(setContextCommand); response = hasCommandExecutor.CommandExecutor.Execute(getContextCommand); Assert.AreEqual("chrome", response.Value); Driver.ExecuteJavaScript("Services.prefs.setStringPref('intl.accept_languages', 'es-ES')"); try { Driver.Navigate().Refresh(); Assert.Fail("Can not navigate in chrome context, so previous method should fail"); } catch (WebDriverException) { // This is expected } payload.Remove("context"); payload.Add("context", "content"); setContextCommand = new Command(sessionId, "setContext", payload); hasCommandExecutor.CommandExecutor.Execute(setContextCommand); Driver.Navigate().Refresh(); element = Driver.FindElement(By.CssSelector("#gws-output-pages-elements-homepage_additional_languages__als")); Assert.IsTrue(element.Text.Contains("Ofrecido por")); ``` ### BiDirectional Functionality The [WebDriver BiDirectional Protocol](https://testingbot.com/support/web-automate/selenium/selenium-bidi) is a new protocol enhancement that should improve the speed and reliability of Webdriver tests. Its mechanism is similar to CDP (Chrome DevTools Protocol), but is implemented as a vendor agnostic protocol. TestingBot supports [WebDriver BiDi](https://testingbot.com/support/web-automate/selenium/selenium-bidi). Please see the [BiDi documentation examples](https://testingbot.com/support/web-automate/selenium/selenium-bidi) to get started. ## Upgrading dependencies Please see the examples below on how to upgrade to Selenium 4. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [JavaScript](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby gem 'selenium-webdriver', '~> 4.36' ``` ```java org.seleniumhq.selenium selenium-java 4.36.0 ``` You can run `npm install selenium-webdriver` to install the latest version, or create/modify your `package.json` file: ```javascript { "name": "selenium-tests", "version": "1.0.0", "dependencies": { "selenium-webdriver": "^4.36.0" } } ``` ```bash pip install 'selenium>=4.36' ``` ```csharp PM> Install-Package Selenium.WebDriver -Version 4.36.0 ``` ## Deprecation Messages and Errors Below is a list of potential errors or deprecation messages you might see when upgrading to Selenium 4. ### AddAdditionalOption The `AddAdditionalCapability` capability is deprecated in Selenium 4, it has been replaced by `AddAdditionalOption`. [Legacy](https://testingbot.com#) [Selenium 4](https://testingbot.com#) ```csharp var browserOptions = new ChromeOptions(); browserOptions.PlatformName = "Windows 10"; browserOptions.BrowserVersion = "latest"; var tbOptions = new Dictionary(); browserOptions.AddAdditionalCapability("tb:options", tbOptions, true); ``` ```csharp var browserOptions = new ChromeOptions(); browserOptions.PlatformName = "Windows 10"; browserOptions.BrowserVersion = "latest"; var tbOptions = new Dictionary(); browserOptions.AddAdditionalOption("tb:options", tbOptions); ``` ### Timeout Parameters The parameters passed to `Timeout` functions have switched from long (`TimeUnit`) to duration (`Duration`). [Legacy](https://testingbot.com#) [Selenium 4](https://testingbot.com#) ```java driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS); driver.manage().timeouts().setScriptTimeout(4, TimeUnit.MINUTES); driver.manage().timeouts().pageLoadTimeout(5, TimeUnit.SECONDS); ``` ```java driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(10)); driver.manage().timeouts().scriptTimeout(Duration.ofMinutes(4)); driver.manage().timeouts().pageLoadTimeout(Duration.ofSeconds(5)); ``` ### Wait Parameters `WebDriverWait`, `withTimeout` and `pollingEvery` now expect `Duration` instead of a long time. [Legacy](https://testingbot.com#) [Selenium 4](https://testingbot.com#) ```java new WebDriverWait(driver, 3) .until(ExpectedConditions.elementToBeClickable(By.cssSelector("#id"))); Wait wait = new FluentWait(driver) .withTimeout(30, TimeUnit.SECONDS) .pollingEvery(5, TimeUnit.SECONDS) .ignoring(NoSuchElementException.class); ``` ```java new WebDriverWait(driver, Duration.ofSeconds(3)) .until(ExpectedConditions.elementToBeClickable(By.cssSelector("#id"))); Wait wait = new FluentWait(driver) .withTimeout(Duration.ofSeconds(30)) .pollingEvery(Duration.ofSeconds(5)) .ignoring(NoSuchElementException.class); ``` ### Merging Capabilities Before Selenium 4 you could merge capabilities, mutating the calling object. Since Selenium 4 this is deprecated, you now need to manually assign the result of the merge operation. [Legacy](https://testingbot.com#) [Selenium 4](https://testingbot.com#) ```java MutableCapabilities capabilities = new MutableCapabilities(); capabilities.setCapability("platformVersion", "Windows 10"); FirefoxOptions options = new FirefoxOptions(); options.addArguments("--headless"); options.merge(capabilities); ``` ```java MutableCapabilities capabilities = new MutableCapabilities(); capabilities.setCapability("platformVersion", "Windows 10"); FirefoxOptions options = new FirefoxOptions(); options.addArguments("--headless"); options = options.merge(capabilities); ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/selenium-bidi # Learn about event-driven testing with Selenium BiDi [WebDriver BiDi](https://w3c.github.io/webdriver-bidi/) is a new protocol, standardized by the W3C. Even though it's still in draft mode, [support for this new protocol](https://wpt.fyi/results/webdriver/tests/bidi?label=master&label=stable&aligned) has already been implemented in Chrome, Microsoft Edge and Firefox. The advantage of using this new protocol (based on WebSocket) over the older one (HTTP) is that it allows for **bidi** rectional communication between the test script and the Selenium endpoint. This means your test scripts can now listen for certain events, generated by the (remote) browser under test. Previously, a similar protocol was developed called [CDP](https://www.selenium.dev/documentation/webdriver/bidi/cdp/) ( **C** hrome **D** evTools **P** rotocol), which only supported Chromium-based browsers (Chrome and Microsoft Edge). The new BiDi protocol is the new official standards-based alternative to CDP. It is supported on multiple browsers and is used for automation with [Puppeteer](https://testingbot.com/support/web-automate/puppeteer). ## Key Features of WebDriver BiDi Protocol There are various advantages to using Selenium BiDi: - **Event-Driven Automation:** Allows your test scripts to listen to browser based events as they happen. This eliminates the need for polling and improves the overall test performance. - **Standardization:** The WebDriver BiDi protocol is a W3C standard, which means you'll get the same consistency and interoperability between browser flavors. You can run the same BiDi test script on multiple different browsers on the TestingBot browser grid. - **Future Proof:** The most popular browser vendors are committed to supporting the WebDriver BiDi Protocol. By adopting BiDi early on, you are future-proofing your test cases and ensuring compatibility with upcoming browser advancements. - **Cross-Browser Compatibility** Unlike CDP, which is restricted to a limited set of browsers, the WebDriver BiDi Protocol is designed to operate across various browsers, including Firefox. ## Configuring Selenium BiDi To get started with Selenium BiDi on TestingBot, you'll need to add several capabilities to your existing Selenium tests: ### Selenium Version 4 is required Selenium BiDi is only available on Selenium 4. We suggest using Selenium version `4.23.0` or higher. Please specify the `selenium-version` capability in a `tb:options` capability. [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) [C#](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) ```ruby caps = { platformName: "Windows 10", browserName: "chrome", browserVersion: "latest", "tb:options" => { "selenium-version" => '4.33.0' } } options.web_socket_url = true ``` ```java ChromeOptions options = new ChromeOptions(); options.setPlatformName("Windows 11"); options.setBrowserVersion("latest"); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("selenium-version", "4.33.0"); options.setCapability("tb:options", tbOptions); ``` ```php $capabilities = DesiredCapabilities::chrome(); $capabilities->setCapability('platformName', 'Windows 11'); $capabilities->setCapability('browserVersion', 'latest'); $capabilities->setCapability('tb:options', array( 'selenium-version' => '4.33.0' )); ``` ```python from selenium import webdriver from selenium.webdriver.chrome.options import Options options = Options() options.browser_version = 'latest' options.platform_name = 'Windows 10' options.set_capability('tb:options', { 'name': 'W3C Sample', 'selenium-version': '4.33.0' }) driver = webdriver.Remote( command_executor='https://key:secret@hub.testingbot.com/wd/hub', options=options) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "platformName": 'Windows 10', "goog:chromeOptions" : { "w3c" : true }, "tb:options": { "key": "api_key", "secret": "api_secret", "selenium-version": '4.33.0' } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "Windows 10", UseSpecCompliantProtocol = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["selenium-version"] = "4.33.0" }; chromeOptions.AddAdditionalOption("tb:options", tbOptions); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` ### webSocketUrl capability Please specify the `webSocketUrl` capability to indicate to Selenium that you want to use the BiDi protocol. [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) [C#](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) ```ruby caps = { platformName: "Windows 10", browserName: "chrome", browserVersion: "latest", "selenium-version": "4.33.0", webSocketUrl: true } options.web_socket_url = true ``` ```java ChromeOptions options = new ChromeOptions(); options.setPlatformName("Windows 11"); options.setBrowserVersion("latest"); options.setCapability("webSocketUrl", true); MutableCapabilities tbOptions = new MutableCapabilities(); tbOptions.setCapability("key", "api_key"); tbOptions.setCapability("secret", "api_secret"); tbOptions.setCapability("selenium-version", "4.33.0"); options.setCapability("tb:options", tbOptions); ``` ```php $capabilities = DesiredCapabilities::chrome(); $capabilities->setCapability('platformName', 'Windows 11'); $capabilities->setCapability('browserVersion', 'latest'); $capabilities->setCapability('webSocketUrl', true); $capabilities->setCapability('tb:options', array( 'selenium-version' => '4.33.0' )); ``` ```python from selenium import webdriver from selenium.webdriver.chrome.options import Options options = Options() options.browser_version = 'latest' options.platform_name = 'Windows 10' options.enable_bidi = True options.set_capability('tb:options', { 'name': 'W3C Sample', 'selenium-version': '4.33.0' }) driver = webdriver.Remote( command_executor='https://key:secret@hub.testingbot.com/wd/hub', options=options) ``` ```javascript driver = await new webdriver.Builder().withCapabilities({ "browserName": 'chrome', "platformName": 'Windows 10', "webSocketUrl": true, "tb:options": { "key": "api_key", "secret": "api_secret", "selenium-version": '4.33.0' } }).usingServer("https://hub.testingbot.com/wd/hub").build(); ``` ```csharp var chromeOptions = new ChromeOptions() { BrowserVersion = "latest", PlatformName = "Windows 10", UseWebSocketUrl = true }; var tbOptions = new Dictionary { ["key"] = "api_key", ["secret"] = "api_secret", ["selenium-version"] = "4.33.0" }; chromeOptions.AddAdditionalOption("tb:options", tbOptions); driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); ``` ## Listen to console.log events This example shows how to use Selenium BiDi to collect events generated by the browser in realtime, such as the `console.log` events. It will navigate to a website and click a button that generates `console.log` entries. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) [Ruby](https://testingbot.com#) ```ruby require "rubygems" require "selenium-webdriver" tb_options = { "name" => "My Bidi Test", "browserName" => "chrome", "platform" => "WIN10", "version" => "latest", "selenium-version" => "4.33.0" } options = Selenium::WebDriver::Chrome::Options.new options.add_option('tb:options', tb_options) options.web_socket_url = true driver = Selenium::WebDriver.for( :remote, :url => "https://key:secret@hub.testingbot.com/wd/hub", :capabilities => options) driver.navigate.to "https://www.selenium.dev/selenium/web/bidi/logEntryAdded.html" driver.script.add_console_message_handler { |log| puts log } wait = Selenium::WebDriver::Wait.new(timeout: 10) consoleLog = wait.until { element = driver.find_element(id: 'consoleLog') element if element.displayed? } consoleLog.click driver.quit ``` ```java import com.testingbot.comingbotrest.TestingbotREST; import java.net.MalformedURLException; import java.net.URL; import java.util.HashMap; import java.util.Map; import org.openqa.selenium.By; import org.openqa.selenium.JavascriptExecutor; import org.openqa.selenium.MutableCapabilities; import org.openqa.selenium.WebDriver; import org.openqa.selenium.bidi.module.LogInspector; import org.openqa.selenium.remote.Augmenter; import org.openqa.selenium.remote.RemoteWebDriver; public class ListenToConsoleLogs { public static final String KEY = "KEY"; public static final String SECRET = "SECRET"; public static final String REMOTE_URL = "https://" + KEY + ":" + SECRET + "@hub.testingbot.com/wd/hub"; static Boolean success = false; public static void main(String[] args) throws MalformedURLException { MutableCapabilities capabilities = new MutableCapabilities(); capabilities.setCapability("browserName", "chrome"); capabilities.setCapability("webSocketUrl", true); HashMap testingbotOptions = new HashMap<>(); testingbotOptions.put("selenium-version", "4.33.0"); capabilities.setCapability("tb:options", testingbotOptions); WebDriver driver = new RemoteWebDriver(new URL(REMOTE_URL), capabilities); try { Augmenter augmenter = new Augmenter(); driver = augmenter.augment(driver); try (LogInspector logInspector = new LogInspector(driver)) { logInspector.onConsoleEntry(consoleLogEntry -> { System.out.println("text: " + consoleLogEntry.getText()); System.out.println("level: " + consoleLogEntry.getLevel()); success = true; }); String page = "https://www.selenium.dev/selenium/web/bidi/logEntryAdded.html"; driver.get(page); driver.findElement(By.id("consoleLog")).click(); if (success) { markTestStatus(true, "Console logs streaming", driver); } else { markTestStatus(false, "Console logs did not stream", driver); } driver.quit(); } } catch (Exception e) { markTestStatus(false, "Exception!", driver); e.printStackTrace(); driver.quit(); } } public static void markTestStatus(Boolean status, String reason, WebDriver driver) { TestingbotREST r = new TestingbotREST(System.getenv("TESTINGBOT_KEY"), System.getenv("TESTINGBOT_SECRET")); Map data = new HashMap(); data.put("success", status ? "1" : "0"); data.put("name", reason); r.updateTest(((RemoteWebDriver) driver).getSessionId().toString(), data); } } ``` ```python from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.support.wait import WebDriverWait def console_log(): tb_options = { 'name': 'TestingBot Sample Test', 'selenium-version': '4.33.0' } options = webdriver.ChromeOptions() options.browser_version = 'latest' options.platform_name = 'Windows 11' options.set_capability('webSocketUrl', True) options.set_capability('tb:options', tb_options) driver = webdriver.Remote( command_executor='https://key:secret@hub.testingbot.com/wd/hub', options=options) driver.get('https://www.selenium.dev/selenium/web/bidi/logEntryAdded.html') log_entries = [] driver.script.add_console_message_handler(log_entries.append) driver.find_element(By.ID, "consoleLog").click() WebDriverWait(driver, 5).until(lambda _: log_entries) print(log_entries[0].text) driver.quit() ``` ```javascript const webdriver = require('selenium-webdriver'); const USERNAME = process.env.TESTINGBOT_KEY; const ACCESS_KEY = process.env.TESTINGBOT_SECRET; (async () => { const driver = await (new webdriver.Builder() .withCapabilities({ browserName: 'chrome', platformName: 'WIN10', webSocketUrl: true, 'tb:options': { 'selenium-version': '4.33.0' }, }) .usingServer( `https://${USERNAME}:${ACCESS_KEY}@hub.testingbot.com/wd/hub` ) .build()); // Add a listener for log events. await driver.script().addConsoleMessageHandler((logEntry) => { console.log(logEntry.text); }); await driver.get( 'https://www.selenium.dev/selenium/web/bidi/logEntryAdded.html' ); // Trigger a console log on the demo page. await driver.findElement({ id: 'consoleLog' }).click(); await driver.quit(); })(); ``` ```csharp using System.Net.WebSockets; using System.Text.Json; using System.Text; using OpenQA.Selenium; using OpenQA.Selenium.BiDi; using OpenQA.Selenium.Remote; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Firefox; using OpenQA.Selenium.Safari; namespace NUnit_TestingBot_Sample { [TestFixture("chrome", "latest", "Windows 10")] [Category("BiDi")] public class BidiTest { private IWebDriver _driver; private CancellationToken _token = CancellationToken.None; private readonly string _browser; private readonly string _version; private readonly string _os; public BidiTest(string browser, string version, string os) { _browser = browser; _version = version; _os = os; } [SetUp] public void SetUp() { var browserOptions = new ChromeOptions { BrowserVersion = _version, PlatformName = _os, UseWebSocketUrl = true }; var testingBotOptions = CreateTestingBotOptions(); browserOptions.AddAdditionalOption("tb:options", testingBotOptions); _driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), browserOptions.ToCapabilities(), TimeSpan.FromSeconds(60) ); } [Test] public async Task CaptureConsoleLogs() { _driver.Navigate().GoToUrl("https://www.selenium.dev/selenium/web/bidi/logEntryAdded.html"); var bidi = await _driver.AsBiDiAsync(); await bidi.Log.OnEntryAddedAsync(e => Console.WriteLine(e)); _driver.FindElement(By.Id("jsException")).Click(); System.Threading.Thread.Sleep(4000); } [TearDown] public void TearDown() { if (_driver == null) return; var testPassed = TestContext.CurrentContext.Result.Outcome.Status == NUnit.Framework.Interfaces.TestStatus.Passed; try { ((IJavaScriptExecutor)_driver).ExecuteScript($"tb:test-result={testPassed.ToString().ToLower()}"); } catch (Exception ex) { TestContext.Out.WriteLine($"Failed to report test result: {ex.Message}"); } finally { _driver.Quit(); _driver.Dispose(); } } private Dictionary CreateTestingBotOptions() { var key = Environment.GetEnvironmentVariable("TESTINGBOT_KEY"); var secret = Environment.GetEnvironmentVariable("TESTINGBOT_SECRET"); if (string.IsNullOrEmpty(key) || string.IsNullOrEmpty(secret)) { throw new InvalidOperationException( "TESTINGBOT_KEY and TESTINGBOT_SECRET environment variables must be set"); } return new Dictionary { ["key"] = key, ["secret"] = secret, ["name"] = TestContext.CurrentContext.Test.Name, ["selenium-version"] = "4.33.0" }; } } } ``` ## Listen to JavaScript Exceptions You can use this API to listen for JavaScript exceptions and register callbacks in your test, to process the exception details. These exception details can help in further inspecting the cause of a failing test. The example test script below will navigate to a web page and click a button that will throw an exception. Selenium BiDi will forward this to your test script. [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) [Ruby](https://testingbot.com#) [CSharp](https://testingbot.com#) ```ruby require "rubygems" require "selenium-webdriver" tb_options = { "name" => "My Bidi Test", "browserName" => "chrome", "platform" => "WIN10", "version" => "latest", "selenium-version" => "4.33.0" } options = Selenium::WebDriver::Chrome::Options.new options.add_option('tb:options', tb_options) options.web_socket_url = true driver = Selenium::WebDriver.for( :remote, :url => "https://key:secret@hub.testingbot.com/wd/hub", :capabilities => options) driver.navigate.to "https://www.selenium.dev/selenium/web/bidi/logEntryAdded.html" driver.script.add_javascript_error_handler { |error| puts error } wait = Selenium::WebDriver::Wait.new(timeout: 10) jsException = wait.until { element = driver.find_element(id: 'jsException') element if element.displayed? } jsException.click driver.quit ``` ```java import com.testingbot.comingbotrest.TestingbotREST; import java.net.MalformedURLException; import java.net.URL; import java.util.HashMap; import java.util.Map; import org.openqa.selenium.By; import org.openqa.selenium.JavascriptExecutor; import org.openqa.selenium.MutableCapabilities; import org.openqa.selenium.WebDriver; import org.openqa.selenium.bidi.module.LogInspector; import org.openqa.selenium.remote.Augmenter; import org.openqa.selenium.remote.RemoteWebDriver; public class ListenToJavaScriptExceptions { public static final String KEY = "KEY"; public static final String SECRET = "SECRET"; public static final String REMOTE_URL = "https://" + KEY + ":" + SECRET + "@hub.testingbot.com/wd/hub"; static Boolean success = false; public static void main(String[] args) throws MalformedURLException { MutableCapabilities capabilities = new MutableCapabilities(); capabilities.setCapability("browserName", "chrome"); capabilities.setCapability("webSocketUrl", true); HashMap testingbotOptions = new HashMap<>(); testingbotOptions.put("selenium-version", "4.33.0"); capabilities.setCapability("tb:options", testingbotOptions); WebDriver driver = new RemoteWebDriver(new URL(REMOTE_URL), capabilities); try { Augmenter augmenter = new Augmenter(); driver = augmenter.augment(driver); try (LogInspector logInspector = new LogInspector(driver)) { logInspector.onJavaScriptException(logEntry -> { System.out.println("text: " + logEntry.getText()); System.out.println("level: " + logEntry.getLevel()); success = true; }); String page = "https://www.selenium.dev/selenium/web/bidi/logEntryAdded.html"; driver.get(page); driver.findElement(By.id("jsException")).click(); Thread.sleep(1500); if (success) { markTestStatus(true, "Caught a JS exception successfully", driver); } else { markTestStatus(false, "Failed to catch the JS exception", driver); } driver.quit(); } } catch (Exception e) { markTestStatus(false, "Exception!", driver); e.printStackTrace(); driver.quit(); } } public static void markTestStatus(Boolean status, String reason, WebDriver driver) { TestingbotREST r = new TestingbotREST(System.getenv("TESTINGBOT_KEY"), System.getenv("TESTINGBOT_SECRET")); Map data = new HashMap(); data.put("success", status ? "1" : "0"); data.put("name", reason); r.updateTest(((RemoteWebDriver) driver).getSessionId().toString(), data); } } ``` ```python from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.support.wait import WebDriverWait def js_exception(): tb_options = { 'name': 'TestingBot Sample Test', 'selenium-version': '4.33.0' } options = webdriver.ChromeOptions() options.browser_version = 'latest' options.platform_name = 'Windows 11' options.set_capability('webSocketUrl', True) options.set_capability('tb:options', tb_options) driver = webdriver.Remote( command_executor='https://key:secret@hub.testingbot.com/wd/hub', options=options) driver.get('https://www.selenium.dev/selenium/web/bidi/logEntryAdded.html') log_entries = [] driver.script.add_javascript_error_handler(log_entries.append) driver.find_element(By.ID, "jsException").click() WebDriverWait(driver, 5).until(lambda _: log_entries) print(log_entries[0].text) driver.quit() ``` ```javascript const webdriver = require('selenium-webdriver'); const USERNAME = process.env.TESTINGBOT_KEY; const ACCESS_KEY = process.env.TESTINGBOT_SECRET; (async () => { const driver = await (new webdriver.Builder() .withCapabilities({ browserName: 'chrome', platformName: 'WIN10', webSocketUrl: true, 'tb:options': { 'selenium-version': '4.33.0' }, }) .usingServer( `https://${USERNAME}:${ACCESS_KEY}@hub.testingbot.com/wd/hub` ) .build()); // Add a listener for error events. await driver.script().addJavaScriptErrorHandler((logEntry) => { console.error(logEntry.text); }); await driver.get( 'https://www.selenium.dev/selenium/web/bidi/logEntryAdded.html' ); // Trigger a console log on the demo page. await driver.findElement({ id: 'jsException' }).click(); await driver.quit(); })(); ``` ```csharp using System.Net.WebSockets; using System.Text.Json; using System.Text; using OpenQA.Selenium; using OpenQA.Selenium.BiDi; using OpenQA.Selenium.Remote; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Firefox; using OpenQA.Selenium.Safari; namespace NUnit_TestingBot_Sample { [TestFixture("chrome", "latest", "Windows 10")] [Category("BiDi")] public class BidiTest { private IWebDriver _driver; private CancellationToken _token = CancellationToken.None; private readonly string _browser; private readonly string _version; private readonly string _os; public BidiTest(string browser, string version, string os) { _browser = browser; _version = version; _os = os; } [SetUp] public void SetUp() { var browserOptions = new ChromeOptions { BrowserVersion = _version, PlatformName = _os, UseWebSocketUrl = true }; var testingBotOptions = CreateTestingBotOptions(); browserOptions.AddAdditionalOption("tb:options", testingBotOptions); _driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), browserOptions.ToCapabilities(), TimeSpan.FromSeconds(60) ); } [Test] public async Task CaptureConsoleLogs() { _driver.Navigate().GoToUrl("https://www.selenium.dev/selenium/web/bidi/logEntryAdded.html"); var bidi = await _driver.AsBiDiAsync(); await bidi.Log.OnEntryAddedAsync(e => Console.WriteLine(e)); _driver.FindElement(By.Id("jsException")).Click(); System.Threading.Thread.Sleep(4000); } [TearDown] public void TearDown() { if (_driver == null) return; var testPassed = TestContext.CurrentContext.Result.Outcome.Status == NUnit.Framework.Interfaces.TestStatus.Passed; try { ((IJavaScriptExecutor)_driver).ExecuteScript($"tb:test-result={testPassed.ToString().ToLower()}"); } catch (Exception ex) { TestContext.Out.WriteLine($"Failed to report test result: {ex.Message}"); } finally { _driver.Quit(); _driver.Dispose(); } } private Dictionary CreateTestingBotOptions() { var key = Environment.GetEnvironmentVariable("TESTINGBOT_KEY"); var secret = Environment.GetEnvironmentVariable("TESTINGBOT_SECRET"); if (string.IsNullOrEmpty(key) || string.IsNullOrEmpty(secret)) { throw new InvalidOperationException( "TESTINGBOT_KEY and TESTINGBOT_SECRET environment variables must be set"); } return new Dictionary { ["key"] = key, ["secret"] = secret, ["name"] = TestContext.CurrentContext.Test.Name, ["selenium-version"] = "4.33.0" }; } } } ``` ## Intercept Network Request You can intercept and modify network requests with Selenium BiDi. Monitor network traffic, or mock outgoing requests. Modify request details such as headers, body, cookies, method and URL. [Java](https://testingbot.com#) [C#](https://testingbot.com#) ```java import com.testingbot.comingbotrest.TestingbotREST; import java.net.MalformedURLException; import java.net.URL; import java.util.HashMap; import java.util.Map; import java.util.concurrent.CountDownLatch; import java.util.concurrent.TimeUnit; import org.openqa.selenium.MutableCapabilities; import org.openqa.selenium.WebDriver; import org.openqa.selenium.bidi.module.Network; import org.openqa.selenium.bidi.network.AddInterceptParameters; import org.openqa.selenium.bidi.network.ContinueRequestParameters; import org.openqa.selenium.bidi.network.InterceptPhase; import org.openqa.selenium.remote.Augmenter; import org.openqa.selenium.remote.RemoteWebDriver; public class InterceptNetworkRequest { public static final String KEY = "KEY"; public static final String SECRET = "SECRET"; public static final String REMOTE_URL = "https://" + KEY + ":" + SECRET + "@hub.testingbot.com/wd/hub"; public static void main(String[] args) throws MalformedURLException { MutableCapabilities capabilities = new MutableCapabilities(); capabilities.setCapability("browserName", "chrome"); capabilities.setCapability("webSocketUrl", true); HashMap testingbotOptions = new HashMap<>(); testingbotOptions.put("selenium-version", "4.33.0"); capabilities.setCapability("tb:options", testingbotOptions); WebDriver driver = new RemoteWebDriver(new URL(REMOTE_URL), capabilities); try { Augmenter augmenter = new Augmenter(); driver = augmenter.augment(driver); try (Network network = new Network(driver)) { network.addIntercept(new AddInterceptParameters(InterceptPhase.BEFORE_REQUEST_SENT)); CountDownLatch latch = new CountDownLatch(1); network.onBeforeRequestSent( beforeRequestSent -> { network.continueRequest( new ContinueRequestParameters(beforeRequestSent.getRequest().getRequestId())); latch.countDown(); }); driver.get("https://www.selenium.dev/selenium/web/bidi/logEntryAdded.html"); boolean countdown = latch.await(5, TimeUnit.SECONDS); if (countdown) { markTestStatus(true, "Network request intercepted successfully", driver); } else { markTestStatus(false, "Network request failed to intercept", driver); } driver.quit(); } } catch (Exception e) { markTestStatus(false, "Exception!", driver); e.printStackTrace(); driver.quit(); } } public static void markTestStatus(Boolean status, String reason, WebDriver driver) { TestingbotREST r = new TestingbotREST(System.getenv("TESTINGBOT_KEY"), System.getenv("TESTINGBOT_SECRET")); Map data = new HashMap(); data.put("success", status ? "1" : "0"); data.put("name", reason); r.updateTest(((RemoteWebDriver) driver).getSessionId().toString(), data); } } ``` ```csharp using System.Net.WebSockets; using System.Text.Json; using System.Text; using OpenQA.Selenium; using OpenQA.Selenium.BiDi; using OpenQA.Selenium.Remote; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Firefox; using OpenQA.Selenium.Safari; namespace NUnit_TestingBot_Sample { [TestFixture("chrome", "latest", "Windows 10")] [Category("BiDi")] public class BidiTest { private IWebDriver _driver; private CancellationToken _token = CancellationToken.None; private readonly string _browser; private readonly string _version; private readonly string _os; public BidiTest(string browser, string version, string os) { _browser = browser; _version = version; _os = os; } [SetUp] public void SetUp() { var browserOptions = new ChromeOptions { BrowserVersion = _version, PlatformName = _os, UseWebSocketUrl = true }; var testingBotOptions = CreateTestingBotOptions(); browserOptions.AddAdditionalOption("tb:options", testingBotOptions); _driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), browserOptions.ToCapabilities(), TimeSpan.FromSeconds(60) ); } [Test] public async Task CaptureConsoleLogs() { _driver.Navigate().GoToUrl("https://www.selenium.dev"); var bidi = await _driver.AsBiDiAsync(); await bidi.Network.InterceptRequestAsync(async req => { if (req.Request.Url.EndsWith(".png")) { await req.ProvideResponseAsync(new() { Body = File.ReadAllBytes("smile.png") }); } else { await req.ContinueAsync(); } }); } [TearDown] public void TearDown() { if (_driver == null) return; var testPassed = TestContext.CurrentContext.Result.Outcome.Status == NUnit.Framework.Interfaces.TestStatus.Passed; try { ((IJavaScriptExecutor)_driver).ExecuteScript($"tb:test-result={testPassed.ToString().ToLower()}"); } catch (Exception ex) { TestContext.Out.WriteLine($"Failed to report test result: {ex.Message}"); } finally { _driver.Quit(); _driver.Dispose(); } } private Dictionary CreateTestingBotOptions() { var key = Environment.GetEnvironmentVariable("TESTINGBOT_KEY"); var secret = Environment.GetEnvironmentVariable("TESTINGBOT_SECRET"); if (string.IsNullOrEmpty(key) || string.IsNullOrEmpty(secret)) { throw new InvalidOperationException( "TESTINGBOT_KEY and TESTINGBOT_SECRET environment variables must be set"); } return new Dictionary { ["key"] = key, ["secret"] = secret, ["name"] = TestContext.CurrentContext.Test.Name, ["selenium-version"] = "4.33.0" }; } } } ``` ## Intercept Network Response You can intercept and modify incoming network responses with Selenium BiDi. Monitor incoming network traffic, or mock incoming requests. Modify request details such as headers, body, cookies, method, URL and status code. [Java](https://testingbot.com#) [C#](https://testingbot.com#) ```java import com.testingbot.comingbotrest.TestingbotREST; import java.net.MalformedURLException; import java.net.URL; import java.util.HashMap; import java.util.Map; import java.util.concurrent.CountDownLatch; import java.util.concurrent.TimeUnit; import org.openqa.selenium.MutableCapabilities; import org.openqa.selenium.WebDriver; import org.openqa.selenium.bidi.module.Network; import org.openqa.selenium.bidi.network.AddInterceptParameters; import org.openqa.selenium.bidi.network.ContinueResponseParameters; import org.openqa.selenium.bidi.network.InterceptPhase; import org.openqa.selenium.remote.Augmenter; import org.openqa.selenium.remote.RemoteWebDriver; public class InterceptNetworkResponse { public static final String KEY = "KEY"; public static final String SECRET = "SECRET"; public static final String REMOTE_URL = "https://" + KEY + ":" + SECRET + "@hub.testingbot.com/wd/hub"; public static void main(String[] args) throws MalformedURLException { MutableCapabilities capabilities = new MutableCapabilities(); capabilities.setCapability("browserName", "chrome"); capabilities.setCapability("webSocketUrl", true); HashMap testingbotOptions = new HashMap<>(); testingbotOptions.put("selenium-version", "4.33.0"); capabilities.setCapability("tb:options", testingbotOptions); WebDriver driver = new RemoteWebDriver(new URL(REMOTE_URL), capabilities); try { Augmenter augmenter = new Augmenter(); driver = augmenter.augment(driver); try (Network network = new Network(driver)) { network.addIntercept(new AddInterceptParameters(InterceptPhase.RESPONSE_STARTED)); CountDownLatch latch = new CountDownLatch(1); network.onResponseStarted( responseDetails -> { network.continueResponse( new ContinueResponseParameters(responseDetails.getRequest().getRequestId())); latch.countDown(); }); driver.get("https://www.selenium.dev/selenium/web/bidi/logEntryAdded.html"); boolean countdown = latch.await(5, TimeUnit.SECONDS); if (countdown) { markTestStatus(true, "Network response intercepted successfully", driver); } else { markTestStatus(false, "Network response failed to intercept", driver); } driver.quit(); } } catch (Exception e) { markTestStatus(false, "Exception!", driver); e.printStackTrace(); driver.quit(); } } public static void markTestStatus(Boolean status, String reason, WebDriver driver) { TestingbotREST r = new TestingbotREST(System.getenv("TESTINGBOT_KEY"), System.getenv("TESTINGBOT_SECRET")); Map data = new HashMap(); data.put("success", status ? "1" : "0"); data.put("name", reason); r.updateTest(((RemoteWebDriver) driver).getSessionId().toString(), data); } } ``` ```csharp using System.Net.WebSockets; using System.Text.Json; using System.Text; using OpenQA.Selenium; using OpenQA.Selenium.BiDi; using OpenQA.Selenium.Remote; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Firefox; using OpenQA.Selenium.Safari; namespace NUnit_TestingBot_Sample { [TestFixture("chrome", "latest", "Windows 10")] [Category("BiDi")] public class BidiTest { private IWebDriver _driver; private CancellationToken _token = CancellationToken.None; private readonly string _browser; private readonly string _version; private readonly string _os; public BidiTest(string browser, string version, string os) { _browser = browser; _version = version; _os = os; } [SetUp] public void SetUp() { var browserOptions = CreateBrowserOptions(); var testingBotOptions = CreateTestingBotOptions(); browserOptions.AddAdditionalOption("tb:options", testingBotOptions); _driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), browserOptions.ToCapabilities(), TimeSpan.FromSeconds(60) ); } [Test] public async Task CaptureNetworkLog() { _driver.Navigate().GoToUrl("https://www.selenium.dev"); var bidi = await _driver.AsBiDiAsync(); await bidi.Network.OnResponseStartedAsync(res => { Console.WriteLine($"URL: {res.Request.Url}"); Console.WriteLine($"Method: {res.Request.Method}"); Console.WriteLine($"Status: {res.Response.Status}"); Console.WriteLine($"Headers: {string.Join(", ", res.Response.Headers.Select(h => $"{h.Name}: {h.Value}"))}"); }); System.Threading.Thread.Sleep(4000); } [TearDown] public void TearDown() { if (_driver == null) return; var testPassed = TestContext.CurrentContext.Result.Outcome.Status == NUnit.Framework.Interfaces.TestStatus.Passed; try { ((IJavaScriptExecutor)_driver).ExecuteScript($"tb:test-result={testPassed.ToString().ToLower()}"); } catch (Exception ex) { TestContext.Out.WriteLine($"Failed to report test result: {ex.Message}"); } finally { _driver.Quit(); _driver.Dispose(); } } private Dictionary CreateTestingBotOptions() { var key = Environment.GetEnvironmentVariable("TESTINGBOT_KEY"); var secret = Environment.GetEnvironmentVariable("TESTINGBOT_SECRET"); if (string.IsNullOrEmpty(key) || string.IsNullOrEmpty(secret)) { throw new InvalidOperationException( "TESTINGBOT_KEY and TESTINGBOT_SECRET environment variables must be set"); } return new Dictionary { ["key"] = key, ["secret"] = secret, ["name"] = TestContext.CurrentContext.Test.Name, ["selenium-version"] = "4.27.0" }; } private DriverOptions CreateBrowserOptions() { return _browser.ToLower() switch { "firefox" => new FirefoxOptions { BrowserVersion = _version, PlatformName = _os, UseWebSocketUrl = true }, "safari" => new SafariOptions { BrowserVersion = _version, PlatformName = _os, UseWebSocketUrl = true }, _ => new ChromeOptions { BrowserVersion = _version, PlatformName = _os, UseWebSocketUrl = true } }; } } } ``` ## Basic Auth By default, Selenium does not support providing an username and password when testing a website behind a Basic Auth protection. The following test example will open a website and enter sample credentials in the basic auth popup window. [Java](https://testingbot.com#) [C#](https://testingbot.com#) ```java import com.testingbot.comingbotrest.TestingbotREST; import java.net.MalformedURLException; import java.net.URL; import java.util.HashMap; import java.util.Map; import org.openqa.selenium.By; import org.openqa.selenium.MutableCapabilities; import org.openqa.selenium.WebDriver; import org.openqa.selenium.bidi.module.Network; import org.openqa.selenium.bidi.network.AddInterceptParameters; import org.openqa.selenium.bidi.network.InterceptPhase; import org.openqa.selenium.remote.Augmenter; import org.openqa.selenium.remote.RemoteWebDriver; import org.openqa.selenium.UsernameAndPassword; public class BasicAuth { public static final String KEY = "KEY"; public static final String SECRET = "SECRET"; public static final String REMOTE_URL = "https://" + KEY + ":" + SECRET + "@hub.testingbot.com/wd/hub"; public static void main(String[] args) throws MalformedURLException { MutableCapabilities capabilities = new MutableCapabilities(); capabilities.setCapability("browserName", "chrome"); capabilities.setCapability("webSocketUrl", true); HashMap testingbotOptions = new HashMap<>(); testingbotOptions.put("selenium-version", "4.33.0"); capabilities.setCapability("tb:options", testingbotOptions); WebDriver driver = new RemoteWebDriver(new URL(REMOTE_URL), capabilities); try { Augmenter augmenter = new Augmenter(); driver = augmenter.augment(driver); try (Network network = new Network(driver)) { network.addIntercept(new AddInterceptParameters(InterceptPhase.AUTH_REQUIRED)); network.onAuthRequired( responseDetails -> network.continueWithAuth( responseDetails.getRequest().getRequestId(), new UsernameAndPassword("foo", "bar"))); driver.get("http://httpbin.org/basic-auth/foo/bar"); String text = driver.findElement(By.tagName("body")).getText(); System.out.println(text); if (text.contains("authenticated")) { markTestStatus(true, "Authentication entered successfully", driver); } else { markTestStatus(false, "Failed to enter authentication credentials", driver); } driver.quit(); } } catch (Exception e) { markTestStatus(false, "Exception!", driver); e.printStackTrace(); driver.quit(); } } public static void markTestStatus(Boolean status, String reason, WebDriver driver) { TestingbotREST r = new TestingbotREST(System.getenv("TESTINGBOT_KEY"), System.getenv("TESTINGBOT_SECRET")); Map data = new HashMap(); data.put("success", status ? "1" : "0"); data.put("name", reason); r.updateTest(((RemoteWebDriver) driver).getSessionId().toString(), data); } } ``` ```csharp using System.Net.WebSockets; using System.Text.Json; using System.Text; using OpenQA.Selenium; using OpenQA.Selenium.BiDi; using OpenQA.Selenium.Remote; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Firefox; using OpenQA.Selenium.Safari; namespace NUnit_TestingBot_Sample { [TestFixture("chrome", "latest", "Windows 10")] [Category("BiDi")] public class BidiTest { private IWebDriver _driver; private CancellationToken _token = CancellationToken.None; private readonly string _browser; private readonly string _version; private readonly string _os; public BidiTest(string browser, string version, string os) { _browser = browser; _version = version; _os = os; } [SetUp] public void SetUp() { var browserOptions = CreateBrowserOptions(); var testingBotOptions = CreateTestingBotOptions(); browserOptions.AddAdditionalOption("tb:options", testingBotOptions); _driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), browserOptions.ToCapabilities(), TimeSpan.FromSeconds(60) ); } [Test] public async Task CaptureNetworkLog() { _driver.Navigate().GoToUrl("https://www.selenium.dev"); var bidi = await _driver.AsBiDiAsync(); await bidi.Network.OnAuthRequiredAsync(async auth => { if (auth.Request.Url.Contains("httpbin.org/basic-auth")) { await auth.ContinueWithAuthAsync(new UsernameAndPassword("foo", "bar")); } }); await bidi.Network.OnResponseStartedAsync(res => { Console.WriteLine($"URL: {res.Request.Url}"); Console.WriteLine($"Method: {res.Request.Method}"); Console.WriteLine($"Status: {res.Response.Status}"); Console.WriteLine($"Headers: {string.Join(", ", res.Response.Headers.Select(h => $"{h.Name}: {h.Value}"))}"); }); System.Threading.Thread.Sleep(4000); } [TearDown] public void TearDown() { if (_driver == null) return; var testPassed = TestContext.CurrentContext.Result.Outcome.Status == NUnit.Framework.Interfaces.TestStatus.Passed; try { ((IJavaScriptExecutor)_driver).ExecuteScript($"tb:test-result={testPassed.ToString().ToLower()}"); } catch (Exception ex) { TestContext.Out.WriteLine($"Failed to report test result: {ex.Message}"); } finally { _driver.Quit(); _driver.Dispose(); } } private Dictionary CreateTestingBotOptions() { var key = Environment.GetEnvironmentVariable("TESTINGBOT_KEY"); var secret = Environment.GetEnvironmentVariable("TESTINGBOT_SECRET"); if (string.IsNullOrEmpty(key) || string.IsNullOrEmpty(secret)) { throw new InvalidOperationException( "TESTINGBOT_KEY and TESTINGBOT_SECRET environment variables must be set"); } return new Dictionary { ["key"] = key, ["secret"] = secret, ["name"] = TestContext.CurrentContext.Test.Name, ["selenium-version"] = "4.27.0" }; } private DriverOptions CreateBrowserOptions() { return _browser.ToLower() switch { "firefox" => new FirefoxOptions { BrowserVersion = _version, PlatformName = _os, UseWebSocketUrl = true }, "safari" => new SafariOptions { BrowserVersion = _version, PlatformName = _os, UseWebSocketUrl = true }, _ => new ChromeOptions { BrowserVersion = _version, PlatformName = _os, UseWebSocketUrl = true } }; } } } ``` ## Intercept Network Response Using CDP (DevTools) you can intercept network responses generated during the Selenium test. [Java](https://testingbot.com#) [C#](https://testingbot.com#) ```java import org.junit.jupiter.api.*; import org.openqa.selenium.*; import org.openqa.selenium.chrome.*; import org.openqa.selenium.devtools.*; // Selenium's typed CDP Network module is version-pinned. Use vNNN matching the Chrome major // you request on the grid (e.g. v140 for Chrome 140). Selenium also auto-falls-back to the // nearest supported CDP version at runtime via Augmenter, but the typed imports below must // match a version included in your selenium-java client. import org.openqa.selenium.devtools.v140.network.Network; import org.openqa.selenium.devtools.v140.network.model.RequestId; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URI; import java.time.Duration; import java.util.Base64; import java.util.HashMap; import java.util.Optional; public class WebDriverDevToolsTest { private RemoteWebDriver driver; private DevTools devTools; @BeforeEach public void setUp() throws Exception { ChromeOptions options = new ChromeOptions(); options.setBrowserVersion("140"); options.setPlatformName("Windows 11"); HashMap tbOptions = new HashMap<>(); tbOptions.put("key", System.getenv("TESTINGBOT_KEY")); tbOptions.put("secret", System.getenv("TESTINGBOT_SECRET")); tbOptions.put("name", "DevTools Network Example"); tbOptions.put("selenium-version", "4.33.0"); tbOptions.put("build", "Java-DevTools-" + System.currentTimeMillis()); tbOptions.put("webSocketUrl", true); options.setCapability("tb:options", tbOptions); driver = new RemoteWebDriver( new URI("https://hub.testingbot.com/wd/hub").toURL(), options ); driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(10)); devTools = ((HasDevTools) driver).getDevTools(); devTools.createSession(); } @Test public void recordNetworkResponses() { devTools.send(Network.enable(Optional.empty(), Optional.empty(), Optional.empty())); devTools.addListener(Network.responseReceived(), response -> { System.out.println("URL: " + response.getResponse().getUrl()); System.out.println("Status: " + response.getResponse().getStatus()); System.out.println("Headers: " + response.getResponse().getHeaders()); RequestId requestId = response.getRequestId(); try { String body = devTools.send(Network.getResponseBody(requestId)).getBody(); boolean isBase64 = devTools.send(Network.getResponseBody(requestId)).getBase64Encoded(); if (isBase64) { body = new String(Base64.getDecoder().decode(body)); } System.out.println("Body: " + body); } catch (Exception e) { System.out.println("Could not fetch response body: " + e.getMessage()); } }); driver.get("https://www.selenium.dev"); try { Thread.sleep(3000); // wait for responses } catch (InterruptedException ignored) {} } @AfterEach public void tearDown() { if (driver != null) { try { ((JavascriptExecutor) driver).executeScript("tb:test-result=true"); } catch (Exception ignored) {} driver.quit(); } } } ``` ```csharp using System.Collections.Generic; using System.Threading.Tasks; using OpenQA.Selenium; using OpenQA.Selenium.Remote; using OpenQA.Selenium.DevTools; using OpenQA.Selenium.Chrome; using System.Linq; namespace NUnit_TestingBot_Sample { [TestFixture("chrome", "137", "Windows 10")] [Category("DevTools")] public class WebDriverDevtoolsTests { private IWebDriver? _driver; private readonly string _browser; private readonly string _version; private readonly string _os; public WebDriverDevtoolsTests(string browser, string version, string os) { _browser = browser; _version = version; _os = os; } [SetUp] public void SetUp() { var browserOptions = CreateBrowserOptions(); var testingBotOptions = CreateTestingBotOptions(); browserOptions.AddAdditionalOption("tb:options", testingBotOptions); _driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), browserOptions.ToCapabilities(), TimeSpan.FromSeconds(600)); _driver.Manage().Timeouts().ImplicitWait = TimeSpan.FromSeconds(10); } [Test] public async Task RecordNetworkResponse() { INetwork networkInterceptor = _driver!.Manage().Network; networkInterceptor.NetworkResponseReceived += (_, e) => { Console.WriteLine($"Response body: {e.ResponseBody}"); Console.WriteLine($"Response headers: {string.Join(", ", e.ResponseHeaders.Select(h => $"{h.Key}: {h.Value}"))}"); Console.WriteLine($"Response status code: {e.ResponseStatusCode}"); Console.WriteLine($"Response URL: {e.ResponseUrl}"); }; await networkInterceptor.StartMonitoring(); _driver!.Navigate().GoToUrl("https://www.selenium.dev/"); await networkInterceptor.StopMonitoring(); } private ChromeOptions CreateBrowserOptions() { var options = new ChromeOptions { BrowserVersion = _version, PlatformName = _os, UseWebSocketUrl = true }; return options; } [TearDown] public void TearDown() { _driver?.Quit(); _driver?.Dispose(); } private Dictionary CreateTestingBotOptions() { var key = Environment.GetEnvironmentVariable("TESTINGBOT_KEY"); var secret = Environment.GetEnvironmentVariable("TESTINGBOT_SECRET"); if (string.IsNullOrEmpty(key) || string.IsNullOrEmpty(secret)) { throw new InvalidOperationException( "TESTINGBOT_KEY and TESTINGBOT_SECRET environment variables must be set"); } return new Dictionary { ["key"] = key, ["secret"] = secret, ["name"] = $"{TestContext.CurrentContext.Test.Name}", ["selenium-version"] = "4.33.0", ["build"] = $"DevTools-Tests-{DateTime.Now:yyyy-MM-dd}" }; } } } ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/oxygen # Oxygen Automated Browser Testing Oxygen is a powerful Selenium & Appium framework. It comes with a dedicated IDE and supports these features: - Selenese - Page Objects - Record, playback and debug functionality ## Installation You can download the Oxygen IDE from [OxygenHQ](https://oxygenhq.org/). You can download either a Windows, macOS or Linux build, depending on the OS you are using. Once installed, you can configure the IDE to use the TestingBot remote browser grid. ## Configuration You can configure the TestingBot integration in Oxygen IDE by opening the `Settings` window, and navigating to `Cloud Providers`. ![Configuring TestingBot integration with Oxygen IDE](https://testingbot.com/assets/support/oxygen/settings-e6951c9595d5d447512c981daf053e019f3f7e5a1bccbae30c48ff435850f519.png) You can then enable the TestingBot integration. Next, you'll need to fill in the `Key` and `Secret` values. ![Configuring TestingBot credentials with Oxygen IDE](https://testingbot.com/assets/support/oxygen/credentials-f5a6d875eb00f92761383719cdc1952355ca92067d93f4606fb8a101c7cdebee.png) ## Run Test Now you're ready to create your first test. Follow the instructions from the Oxygen IDE/documentation pages, and once you've created your first test, we can run it on a TestingBot browser: ![Run TestingBot test with Oxygen IDE](https://testingbot.com/assets/support/oxygen/run-2557a307b8a1d15e4cd935b2d9c76f755df6a906cb9f4508c54fefc05301a6de.png) ## More Information More information regarding Oxygen IDE is available on the [OxygenHQ website](https://oxygenhq.org/). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/status # Setting Name and Status Each test that you'll run on TestingBot will show up in the [Members Dashboard](https://testingbot.com/members). By default, we do not know what the name of this test is and if it passed or not. To correctly display the test name and status, there are two options: - [use our REST API](https://testingbot.com#api) - [use JavascriptExecutor](https://testingbot.com#javascript_executor) ## TestingBot REST API You can use our [REST API](https://testingbot.com/support/api/tests#updatetest) to update the status and meta-data of a test. The API can be used with any HTTP client, or use one of our [API client libraries](https://testingbot.com/support/api#clients). To use the API, you need the unique identifier of the test that you want to update. Please see [getting the SessionID](https://testingbot.com/support/web-automate/selenium/get-session-id) for more information on how to obtain the sessionID of a specific test. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [cURL](https://testingbot.com#) ```java TestingbotREST restApi = new TestingbotREST("API_KEY", "API_SECRET"); Map details = new HashMap<>(); details.put("name", "MyFirstTest"); details.put("success", true); boolean success = restApi.updateTest(webdriverSessionId, details); ``` ```python from testingbotclient import TestingBotClient tb = TestingBotClient('API_KEY', 'API_SECRET') tb.tests.update_test(webdriver_session_id, name='MyFirstTest', passed=True, build='build-123') ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: 'API_KEY', api_secret: 'API_SECRET' }); api.updateTest({ 'test[success]': '1', 'test[name]': 'MyFirstTest' }, webdriverSessionId, (error, testDetails) => { if (error) console.error(error); }); ``` ```ruby require 'testingbot' api = TestingBot::Api.new('API_KEY', 'API_SECRET') api.update_test(webdriver_session_id, { name: 'MyFirstTest', success: true }) ``` ```php $api = new TestingBot\TestingBotAPI('API_KEY', 'API_SECRET'); $api->updateJob($webdriverSessionId, ['name' => 'MyFirstTest', 'success' => true]); ``` ```bash curl "https://api.testingbot.com/v1/tests/:webdriver_session_id" \ -X PUT \ -d "test[success]=1" \ -d "test[name]=MyFirstTest" \ -u API_KEY:API_SECRET ``` Arguments: - (string) **id** - a unique string identifying your test (= Selenium sessionId, a UUID) - (boolean) **test[success]** - indicating if the test was successful - (string) **test[status\_message]** - the status/error message of your test - (string) **test[name]** - the name of your test - (string) **test[extra]** - extra data for this test - (string) **groups=...,...** - a comma-separated string of groups for this test - (string) **build=...** - what build this test belongs to ## JavascriptExecutor TestingBot has its own, custom JavascriptExecutor methods that you can call during your test, to change the meta-data of the test that is running. Besides changing the name and status of a test, we offer [other JavascriptExecutor methods](https://testingbot.com/support/web-automate/selenium/annotating-tests#start) as well. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java ((JavascriptExecutor)driver).executeScript("tb:test-name=My test"); ((JavascriptExecutor)driver).executeScript("tb:test-result=passed"); ``` ```python driver.execute_script('tb:test-name=My test') driver.execute_script('tb:test-result=passed') ``` ```javascript await driver.executeScript('tb:test-name=My test'); await driver.executeScript('tb:test-result=passed'); ``` ```ruby driver.execute_script('tb:test-name=My test') driver.execute_script('tb:test-result=passed') ``` ```php $driver->executeScript('tb:test-name=My test'); $driver->executeScript('tb:test-result=passed'); ``` ```csharp ((IJavaScriptExecutor)driver).ExecuteScript("tb:test-name=My test"); ((IJavaScriptExecutor)driver).ExecuteScript("tb:test-result=passed"); ``` Available JavascriptExecutor commands: - **tb:test-name=...** - set the name of your test - **tb:test-result=passed|failed** - set whether the test passed or failed - **tb:test-build=...** - what build this test belongs to ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/debug-tests # Debugging Tests TestingBot provides a range of features that will help you with detecting, debugging and fixing of bugs in your Automated Tests. The type of bugs that your Automated Tests may contain include: - **Incorrect selector** : you are trying to find an element on the page that is no longer there, or has changed its `className`, `id` or `xPath` - **Missing/Incorrect wait condition** : you need to wait for an element to appear, or are using the wrong wait approach - **Timeouts** : perhaps you forgot to close the session at the end of your test (`driver.quit`) - **Other bugs** : various other bugs that might affect your test quality TestingBot provides various tools to help you with these problems: - [Interactive Session](https://testingbot.com#interactive): interact with the VM or device while your test is running - [Video](https://testingbot.com#video): see a video recording of your test - [Logs](https://testingbot.com#logs): a collection of logs is available for each test. ## Interactive Session Our interactive session feature allows you to pause a running Selenium or Appium script, and inspect the remote browser or app with the included Developer Tools. Simply click the **Pause Test** button on the test page while the test is running and you will be navigated to a remote session. You can now use your mouse and keyboard to debug potential issues. ## Video Recording Every test that runs on TestingBot comes with a [recorded video](https://testingbot.com/support/web-automate/selenium/test-options#screenrecorder) of the test session. The video is available in our member area, where you navigate to a test detail page. You can also [fetch the video](https://testingbot.com/support/api/tests#singletest), together with other details, through the TestingBot REST API. ## Log Files For every test that runs on TestingBot, we collect logs generated by the test session. These logs can be useful for pinpointing certain issues with your tests. Log files are available on the test detail pages. You can also [fetch the logs](https://testingbot.com/support/api/tests#singletest), together with other details, through the TestingBot REST API. For the errors themselves, see our [Selenium WebDriver exceptions](https://testingbot.com/resources/articles/exception-handling-with-selenium) article, and [ElementClickInterceptedException in Selenium](https://testingbot.com/resources/articles/selenium-elementclickinterceptedexception) for the click failure that is hardest to tell apart from the others. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/get-session-id # Get Selenium WebDriver SessionID The Selenium WebDriver SessionID is a unique identifier for each of your tests. Usually it's returned in a [UUID format](https://www.w3.org/TR/webdriver/#dfn-generating-a-uuid). You can use this unique identifier for your own purposes, or to use it in combination with the [TestingBot API](https://testingbot.com/support/api). Please see the examples below on how to retrieve the SessionID: [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java RemoteWebDriver driver = new RemoteWebDriver(new URL(URL), options); String sessionId = driver.getSessionId().toString(); ``` ```python session_id = driver.session_id ``` ```javascript const session = await driver.getSession(); const sessionId = session.getId(); ``` ```ruby session_id = driver.session_id ``` ```php $sessionId = $driver->getSessionID(); ``` ```csharp string sessionId = ((RemoteWebDriver)driver).SessionId.ToString(); ``` ## Using the Selenium SessionID You can use the Selenium SessionID to [mark a test as passed or failed](https://testingbot.com/support/web-automate/selenium/status). You can also use the WebDriver sessionID to link to a test or [share a test](https://testingbot.com/support/other/sharing). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/insecure-certificates # Accepting Insecure Certificates You might want to test a website that does not (yet) have a valid SSL certificate (either it's expired, self-signed, or other). We recommend using our [TestingBot Tunnel](https://testingbot.com/support/tunnel) for this. The Tunnel will automatically convert the invalid certificate to a certificate that all TestingBot VMs and Devices trust. You can also use the W3C `acceptInsecureCerts` capability (set it to `true` on your browser options), but it does not work as reliably as the TestingBot Tunnel approach. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/proxy # Run tests from behind a proxy If the machine you are using to run tests is behind a proxy, chances are that you won't be able to connect to TestingBot's grid. In this case, you can instruct Selenium to go through your proxy. See the example below. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java // For HTTP System.getProperties().put("http.proxyHost", "proxy.example.com"); System.getProperties().put("http.proxyPort", "8080"); System.getProperties().put("http.proxyUser", "username"); System.getProperties().put("http.proxyPassword", "password"); // For HTTPS System.getProperties().put("https.proxyHost", "proxy.example.com"); System.getProperties().put("https.proxyPort", "8080"); System.getProperties().put("https.proxyUser", "username"); System.getProperties().put("https.proxyPassword", "password"); ``` ```python import os # Set proxy environment variables before creating the driver os.environ['HTTP_PROXY'] = 'http://proxy.example.com:8080' os.environ['HTTPS_PROXY'] = 'http://proxy.example.com:8080' # Or with authentication os.environ['HTTP_PROXY'] = 'http://username:password@proxy.example.com:8080' os.environ['HTTPS_PROXY'] = 'http://username:password@proxy.example.com:8080' ``` ```javascript const { Builder } = require('selenium-webdriver'); const { HttpsProxyAgent } = require('https-proxy-agent'); const proxyAgent = new HttpsProxyAgent('http://proxy.example.com:8080'); const driver = new Builder() .usingHttpAgent(proxyAgent) .usingServer('https://hub.testingbot.com/wd/hub') .withCapabilities(capabilities) .build(); ``` ```ruby # Set proxy environment variables ENV['http_proxy'] = 'http://proxy.example.com:8080' ENV['https_proxy'] = 'http://proxy.example.com:8080' # Or with authentication ENV['http_proxy'] = 'http://username:password@proxy.example.com:8080' ENV['https_proxy'] = 'http://username:password@proxy.example.com:8080' ``` ```php $proxyHost = 'proxy.example.com'; $proxyPort = 8080; $driver = RemoteWebDriver::create( 'https://hub.testingbot.com/wd/hub', $caps, 120000, // connection timeout 120000, // request timeout $proxyHost, $proxyPort ); ``` ```csharp var proxy = new WebProxy("proxy.example.com", 8080); var commandExecutor = new HttpCommandExecutor( new Uri("https://hub.testingbot.com/wd/hub"), TimeSpan.FromSeconds(60) ); commandExecutor.Proxy = proxy; ChromeOptions options = new ChromeOptions(); options.PlatformName = "WIN11"; options.BrowserVersion = "latest"; var tbOptions = new Dictionary { ["key"] = "API_KEY", ["secret"] = "API_SECRET" }; options.AddAdditionalOption("tb:options", tbOptions); IWebDriver driver = new RemoteWebDriver(commandExecutor, options); ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/screenshots # Taking screenshots During your Automated Testing, you can take screenshots of the current webpage by issuing a "Take Screenshot" command. Depending on which framework/language binding you are using, the syntax might be a little bit different. If you want to automatically take a screenshot for each step in your test, you can use the custom [screenshot TestingBot capability](https://testingbot.com/support/web-automate/selenium/test-options#screenshot). [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby driver.save_screenshot("screenshot.png") ``` ```java File srcFile = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE); FileUtils.copyFile(srcFile, new File("/location/to/screenshot.png")); ``` ```php file_put_contents('screenshot.png', $web_driver->takeScreenshot()); ``` ```python driver.save_screenshot('screenshot.png') ``` ```javascript const fs = require('fs'); const data = await driver.takeScreenshot(); fs.writeFileSync('screenshot.png', data, 'base64'); ``` ```csharp Screenshot screenshot = ((ITakesScreenshot)driver).GetScreenshot(); screenshot.SaveAsFile("screenshot.png"); ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/sensitive-data # Hiding Sensitive Data When you are running tests on the TestingBot cloud, we automatically gather logs and screenshots/videos of each of your tests. This feature allows testers to debug potential issues with their tests, or verify if their tests are doing what they are supposed to be doing. However, it might be important for you to prevent this logging from happening. For example, you're testing a login/signup page and you are sending a password or secret token. We offer several options to partially or completely disable logging during your tests: - [stripping parameters in logs](https://testingbot.com#params) - [completely disabling logs](https://testingbot.com#disable) - [disable video recordings](https://testingbot.com#video) The capabilities below (`recordLogs`, `disableLogs`, `screenrecorder`) are TestingBot-specific. In Selenium 4 / W3C mode they must be nested inside the `tb:options` capability, see the example below, not set as top-level keys. ```java // Selenium 4 / W3C example - nest TestingBot custom keys inside tb:options ChromeOptions options = new ChromeOptions(); Map tbOptions = new HashMap<>(); tbOptions.put("recordLogs", "strip-parameters"); tbOptions.put("disableLogs", new String[] {"appium", "browser"}); tbOptions.put("screenrecorder", false); options.setCapability("tb:options", tbOptions); ``` ## Strip Parameters You can choose to remove all parameters from the TestingBot logs, which are sent during your test. Parameters like passwords, tokens or other sensitive data will no longer appear in our logs. [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps["recordLogs"] = "strip-parameters" ``` ```java caps.setCapability("recordLogs", "strip-parameters"); ``` ```php $caps["recordLogs"] = "strip-parameters"; ``` ```python caps["recordLogs"] = "strip-parameters" ``` ```javascript capabilities['recordLogs'] = 'strip-parameters' ``` ```csharp caps.SetCapability("recordLogs", "strip-parameters"); ``` ## Disable Logging By default, TestingBot collects several logs during your test. Depending on the browser/platform you're testing on, the log files may include: - **vm** : our logs where we display what the VM is doing, including the Selenium commands it's relaying. - **selenium** : the log file generated by Selenium, which includes parameters and commands your test is sending. - **firefox** : the geckodriver (verbose) log for the test, if the test is running against a Firefox browser. - **sikuli** : if you are using [Sikuli](https://testingbot.com/support/other/sikuli), the output of Sikuli will appear here. - **appium** : mobile tests (live and automated) are using Appium. Commands and parameters will be logged here. - **logcat** : if you are running a mobile app test on Android, we will retrieve the [Android Logcat](https://developer.android.com/studio/command-line/logcat) file. - **android** : this contains the output of the Android emulator. - **browser** : the console logs of the browser. - **cypress** : output generated by Cypress, if you are using [Cypress Testing](https://testingbot.com/support/web-automate/cypress). To **disable all logs** , you can use this capability: [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps["recordLogs"] = false ``` ```java caps.setCapability("recordLogs", false); ``` ```php $caps["recordLogs"] = false; ``` ```python caps["recordLogs"] = False ``` ```javascript capabilities['recordLogs'] = false ``` ```csharp caps.SetCapability("recordLogs", false); ``` You can also choose to **disable some logs** : [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps["disableLogs"] = ['appium', 'browser'] ``` ```java caps.setCapability("disableLogs", new String[] {"appium", "browser"}); ``` ```php $caps["disableLogs"] = array('appium', 'browser'); ``` ```python caps["disableLogs"] = ['appium', 'browser'] ``` ```javascript capabilities['disableLogs'] = ['appium', 'browser'] ``` ```csharp caps.SetCapability("disableLogs", new[] {"appium", "browser"}); ``` ## Disable video recordings By default, we'll record the screen during your test. This makes it easy to troubleshoot or verify the test. You can choose to disable the video recording with this capability: [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby caps["screenrecorder"] = false ``` ```java caps.setCapability("screenrecorder", false); ``` ```php $caps["screenrecorder"] = false; ``` ```python caps["screenrecorder"] = False ``` ```javascript capabilities['screenrecorder'] = false ``` ```csharp caps.SetCapability("screenrecorder", false); ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/test-results # Test Results After running a test on TestingBot, you can see the test result in the [TestingBot Dashboard](https://testingbot.com#dashboard) overview. You can click each test to see a [detailed overview](https://testingbot.com#single) for each specific test. ## The Dashboard The dashboard contains an overview of all your previous test runs, together with some meta-data: - **Test Name** : you can pass this name via our [REST API or Javascript Executor](https://testingbot.com/support/web-automate/selenium/status). - **Pass/Failure state** : you can [indicate](https://testingbot.com/support/web-automate/selenium/status) whether this test passed or failed. - **Browser/Device** : the browser or device that was used by the test. TestingBot provides [more than 5000 different browser/device combinations](https://testingbot.com/support/web-automate/browsers). - **Start Date** : when we received the call to start your test. - **Test Duration** : how long it took for your test to run. This is the actual run time, potential queueing time is not included. ## Single Test Detail Page For each test, we have a test detail page, which shows an overview of all the data we have: - Test Name - [sessionID](https://testingbot.com/support/web-automate/selenium/get-session-id) of the test - the pass/failure state - the browser/device that was used by the test - the date when the test started - how long the test took to run - logs for the test - a video recording of the test - the user who started the test in your team - a list of steps, explaining what the test did ## Viewing results without the dashboard You can [share, embed and link to builds or individual tests](https://testingbot.com/support/other/sharing) without having to visit the dashboard. This is useful when you want to create your own reporting. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/browser-window-size # Maximize or resize browser window Selenium WebDriver makes it possible to resize and maximize a browser window during an automated test. Below you'll find examples on how to: - [Maximize a window](https://testingbot.com#maximize) - [Resize a window](https://testingbot.com#resize) ## Maximize a window The example below shows how to maximize the window to the full screen resolution (you can [change the screen resolution](https://testingbot.com/support/web-automate/selenium/change-screen-resolution) as well). [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java driver.manage().window().maximize(); ``` ```python driver.maximize_window() ``` ```javascript await driver.manage().window().maximize(); ``` ```ruby driver.manage.window.maximize ``` ```php $web_driver->manage()->window()->maximize(); ``` ```csharp driver.Manage().Window.Maximize(); ``` Maximizing a window will only work on desktop browsers, not on mobile. ## Resize a window The example below shows how to resize the current window. If you specify a size larger than the current resolution, elements may appear off-screen. Find out how to [change the screen resolution](https://testingbot.com/support/web-automate/selenium/change-screen-resolution). [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java Dimension dimension = new Dimension(1920, 1080); driver.manage().window().setSize(dimension); ``` ```python driver.set_window_size(1920, 1080) ``` ```javascript await driver.manage().window().setRect({ width: 1920, height: 1080 }); ``` ```ruby driver.manage.window.resize_to(1920, 1080) ``` ```php $web_driver->manage()->window()->setSize(new WebDriverDimension(1920, 1080)); ``` ```csharp driver.Manage().Window.Size = new Size(1920, 1080); ``` Resizing a window will only work on desktop browsers, not on mobile. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/change-screen-resolution # Change desktop screen resolution with Selenium The TestingBot Desktop VMs (Windows, macOS and Linux) all support changing the screen resolution before a test starts. We currently support the following screen resolutions: Value Type | Default Value | || string | "1280x1024" | Platform \| Resolutions \|\| Windows/Linux \| - 800x600 - 1024x768 - 1152x864 - 1280x768 - 1280x800 - 1280x960 - 1280x1024 - 1400x1050 - 1600x1200 - 1680x1050 - 1920x1080 - 1920x1200 - 2560x1440 \| \| macOS \| - 800x600 - 1024x768 - 1280x768 - 1280x800 - 1280x960 - 1280x1024 - 1366x768 - 1440x900 - 1600x900 - 1600x1200 - 1680x1050 - 1920x1080 - 1920x1200 - 2048x1536 \| | [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java ChromeOptions options = new ChromeOptions(); options.setCapability("platformName", "WIN11"); options.setCapability("browserVersion", "latest"); options.setCapability("screen-resolution", "1920x1080"); HashMap tbOptions = new HashMap<>(); tbOptions.put("key", "API_KEY"); tbOptions.put("secret", "API_SECRET"); options.setCapability("tb:options", tbOptions); ``` ```python from selenium.webdriver.chrome.options import Options as ChromeOptions options = ChromeOptions() options.set_capability('platformName', 'WIN11') options.set_capability('browserVersion', 'latest') options.set_capability('screen-resolution', '1920x1080') options.set_capability('tb:options', { 'key': 'API_KEY', 'secret': 'API_SECRET' }) ``` ```javascript const capabilities = { platformName: 'WIN11', browserName: 'chrome', browserVersion: 'latest', 'screen-resolution': '1920x1080', 'tb:options': { key: 'API_KEY', secret: 'API_SECRET' } }; ``` ```ruby options = Selenium::WebDriver::Chrome::Options.new options.add_option('platformName', 'WIN11') options.add_option('browserVersion', 'latest') options.add_option('screen-resolution', '1920x1080') options.add_option('tb:options', { 'key' => 'API_KEY', 'secret' => 'API_SECRET' }) ``` ```php $capabilities = [ 'platformName' => 'WIN11', 'browserName' => 'chrome', 'browserVersion' => 'latest', 'screen-resolution' => '1920x1080', 'tb:options' => [ 'key' => 'API_KEY', 'secret' => 'API_SECRET' ] ]; ``` ```csharp ChromeOptions options = new ChromeOptions(); options.PlatformName = "WIN11"; options.BrowserVersion = "latest"; options.AddAdditionalOption("screen-resolution", "1920x1080"); var tbOptions = new Dictionary { ["key"] = "API_KEY", ["secret"] = "API_SECRET" }; options.AddAdditionalOption("tb:options", tbOptions); ``` The browser window is not maximized when starting a Selenium test. Changing the screen resolution does not maximize the browser. See how to [maximize and resize the browser window](https://testingbot.com/support/web-automate/selenium/browser-window-size). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/basic-http-authentication # Basic HTTP Authentication If you are testing a website that is protected with Basic HTTP Authentication, then your automated test might get stuck when opening the URL. There are two approaches to handle this: the modern [Selenium 4 BiDi approach](https://testingbot.com#selenium4-bidi) (recommended) or the [legacy URL credentials approach](https://testingbot.com#url-credentials). ## URL Credentials (Legacy Approach) You can pass the `username` and `password` in the URL. Note that this approach exposes credentials in browser history and logs. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java driver.get("https://:@www.example.com/password.html"); ``` ```python driver.get("https://:@www.example.com/password.html") ``` ```javascript await driver.get("https://:@www.example.com/password.html"); ``` ```ruby driver.navigate.to "https://:@www.example.com/password.html" ``` ```php $web_driver->get("https://:@www.example.com/password.html"); ``` ```csharp driver.Navigate().GoToUrl("https://:@www.example.com/password.html"); ``` ## Selenium 4 with BiDi (Recommended) Selenium 4 introduced the [BiDi protocol](https://testingbot.com/support/web-automate/selenium/selenium-bidi) which provides a cleaner way to handle Basic Authentication without exposing credentials in the URL. This approach intercepts the authentication request and programmatically provides the credentials, keeping them out of browser history and logs. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [C#](https://testingbot.com#) ```java import org.openqa.selenium.WebDriver; import org.openqa.selenium.bidi.module.Network; import org.openqa.selenium.bidi.network.AddInterceptParameters; import org.openqa.selenium.bidi.network.InterceptPhase; import org.openqa.selenium.remote.Augmenter; import org.openqa.selenium.remote.RemoteWebDriver; import org.openqa.selenium.UsernameAndPassword; // Enable BiDi with webSocketUrl capability capabilities.setCapability("webSocketUrl", true); WebDriver driver = new RemoteWebDriver(new URL(URL), capabilities); driver = new Augmenter().augment(driver); try (Network network = new Network(driver)) { network.addIntercept(new AddInterceptParameters(InterceptPhase.AUTH_REQUIRED)); network.onAuthRequired( responseDetails -> network.continueWithAuth( responseDetails.getRequest().getRequestId(), new UsernameAndPassword("username", "password"))); driver.get("https://httpbin.org/basic-auth/user/passwd"); } ``` ```python from selenium import webdriver from selenium.webdriver.common.bidi.network import Network options = webdriver.ChromeOptions() options.enable_bidi = True options.set_capability('tb:options', { 'selenium-version': '4.33.0' }) driver = webdriver.Remote( command_executor='https://key:secret@hub.testingbot.com/wd/hub', options=options) # Initialize BiDi network module and add authentication handler network = Network(driver) network.add_authentication_handler( username='username', password='password' ) driver.get('https://httpbin.org/basic-auth/user/passwd') driver.quit() ``` ```javascript const { Builder } = require('selenium-webdriver'); const driver = await new Builder() .withCapabilities({ browserName: 'chrome', webSocketUrl: true, 'tb:options': { 'selenium-version': '4.33.0' } }) .usingServer('https://key:secret@hub.testingbot.com/wd/hub') .build(); await driver.network().addAuthenticationHandler('username', 'password'); await driver.get('https://httpbin.org/basic-auth/user/passwd'); await driver.quit(); ``` ```ruby require 'selenium-webdriver' options = Selenium::WebDriver::Chrome::Options.new options.web_socket_url = true options.add_option('tb:options', { 'selenium-version' => '4.33.0' }) driver = Selenium::WebDriver.for( :remote, url: 'https://key:secret@hub.testingbot.com/wd/hub', capabilities: options) driver.register(username: 'username', password: 'password') driver.navigate.to 'https://httpbin.org/basic-auth/user/passwd' driver.quit ``` ```csharp using OpenQA.Selenium; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Remote; var options = new ChromeOptions(); options.AddAdditionalOption("webSocketUrl", true); options.AddAdditionalOption("tb:options", new Dictionary { ["selenium-version"] = "4.33.0" }); var driver = new RemoteWebDriver( new Uri("https://key:secret@hub.testingbot.com/wd/hub"), options.ToCapabilities()); INetwork network = driver.Manage().Network; NetworkAuthenticationHandler handler = new NetworkAuthenticationHandler() { UriMatcher = (uri) => true, Credentials = new PasswordCredentials("username", "password") }; network.AddAuthenticationHandler(handler); await network.StartMonitoring(); driver.Navigate().GoToUrl("https://httpbin.org/basic-auth/user/passwd"); await network.StopMonitoring(); driver.Quit(); ``` For more BiDi examples, see our [Selenium BiDi documentation](https://testingbot.com/support/web-automate/selenium/selenium-bidi). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/browser-extension # Browser Extension While running Automated tests, you might want to have a browser extension installed. Below you can find information on how to add a browser extension to a Chrome, Firefox, Edge or Safari browser running in the TestingBot cloud. To get started, make sure you have the extension file ready, then upload it either to a public URL or use the [TestingBot Storage](https://testingbot.com#storage). Once you have a URL, you can specify the `load-extension` capability. This will instruct the remote TestingBot browsers to download the extension and add it to the browser at startup. Example: ```javascript "load-extension" : "https://..." ``` Example (Selenium 4): ```javascript "tb:options" : { "load-extension" : "https://..." } ``` | Browser | Supported Format(s) | Example | | --- | --- | --- | | [Chrome](https://testingbot.com#chrome) | - `.zip` file, containing the extension source code (`manifest.json`, `src/ directory`, ...) - `.crx` file | - `https://.../extension.zip` - `tb://sample-extension` | | [Firefox](https://testingbot.com#firefox) | - `.zip` file, containing the extension source code (`manifest.json`, `src/ directory`, `lib/ directory`, ...) - `.xpi` file | - `https://.../extension.zip` - `tb://sample-extension` | | [Edge](https://testingbot.com#edge) | - `.zip` file, containing the extension source code (`manifest.json`, `src/ directory`, ...) - `.crx` file | - `https://.../extension.zip` - `tb://sample-extension` | | [Safari & Safari Technology Preview](https://testingbot.com#safari) | - Test (Unsigned) Safari Web/App Extensions bundled in a zip file - `.safariextz` file for Safari \<= 12 | - `https://.../extension.zip` - `tb://sample-extension` | ## TestingBot Storage With [TestingBot Storage](https://testingbot.com/support/api/storage#upload), you can upload your extensions to the TestingBot servers. The advantage of this is that our test VMs can immediately download your extension from the TestingBot network, which is much faster than downloading from the public internet. The API call will return a unique identifier, similar to `tb://efe932jfk`. You can then use this identifier with the capability: `"load-extension" : "tb://..."` ## Chrome Browser Extension Testing Once you've specified `"load-extension"`, the Chrome browser will start up with the extension preloaded. You can now run your Selenium tests while the extension is active on the remote browser. ### UI Testing Chrome Extensions To test the UI of your Chrome extension, you can have your Selenium test navigate to a `chrome-extension://` URL. For example, if the ID of your addon is `paaamahlcokbnafoiahanekijojkhbdm`, you can navigate to `chrome-extension://paaamahlcokbnafoiahanekijojkhbdm`. Please see the NodeJS example below for a complete rundown on how to test the UI of your Chrome extension: ```javascript const webdriver = require('selenium-webdriver'); const capabilities = { 'platformName': 'WIN10', 'browserName': 'chrome', 'browserVersion': 'latest', 'tb:options': { 'load-extension': 'https://...path..to..extension' } }; async function runTest () { let driver = new webdriver.Builder() .usingServer('https://api_key:api_secret@hub.testingbot.com/wd/hub') .withCapabilities(capabilities) .build(); await driver.get('chrome-extension://paaamahlcokbnafoiahanekijojkhbdm/popup/index.html'); const element = await driver.findElement(By.css('#environments')); const isElementDisplayed = await element.isDisplayed(); if (isElementDisplayed) { console.log('Element is present on the page.'); } else { console.log('Element is not present on the page.'); } await driver.quit(); } runTest(); ``` ## Firefox Browser Extension Testing You can specify the URL to a `xpi` file in the `load-extension` capability. This will make sure the Firefox browser has the extension installed (as a temporary extension) at the start of the Selenium test. ### UI Testing Firefox Extensions If you want to test the UI of your Firefox extension, the `moz-extension://` URL, you will need to specify the extension ID in the URL. Because TestingBot installs the extension as a temporary extension, the id will always be different as it is generated as a random uuid(4). TestingBot provides a capability to hardcode your Firefox exension to a fixed uuid which you've generated, by using the `firefoxExtensionMap` capability. To use this, you could set the ID in the `manifest.json` of the extension: ```json "browser_specific_settings": { "gecko": { "id": "addon@testingbot.com", "strict_min_version": "109.0" } } ``` You can now link the gecko id to a uuid you've generated: ```javascript const webdriver = require('selenium-webdriver'); const capabilities = { 'platformName': 'WIN10', 'browserName': 'firefox', 'browserVersion': 'latest', 'tb:options': { 'load-extension': 'https://...path..to..extension', 'firefoxExtensionMap': { 'addon@testingbot.com': '3d9b1639-77fb-44a1-888a-6d97d773e96b' } } }; async function runTest () { let driver = new webdriver.Builder() .usingServer('https://api_key:api_secret@hub.testingbot.com/wd/hub') .withCapabilities(capabilities) .build(); await driver.get('moz-extension://3d9b1639-77fb-44a1-888a-6d97d773e96b/popup/index.html'); const element = await driver.findElement(By.css('#environments')); const isElementDisplayed = await element.isDisplayed(); if (isElementDisplayed) { console.log('Element is present on the page.'); } else { console.log('Element is not present on the page.'); } await driver.quit(); } runTest(); ``` ## Edge Browser Extension Testing Once you've specified `"load-extension"`, the Edge browser will start up with the extension preloaded. You can now run your Selenium tests while the extension is active on the remote Edge browser. ### UI Testing Edge Extensions To test the UI of your Edge extension, you can have your Selenium test navigate to a `chrome-extension://` or `extension://` URL. For example, if the ID of your addon is `paaamahlcokbnafoiahanekijojkhbdm`, you can navigate to `extension://paaamahlcokbnafoiahanekijojkhbdm`. Please see the NodeJS example below for a complete rundown on how to test the UI of your Edge extension: ```javascript const webdriver = require('selenium-webdriver'); const capabilities = { 'platformName': 'WIN10', 'browserName': 'microsoftedge', 'browserVersion': 'latest', 'tb:options': { 'load-extension': 'https://...path..to..extension' } }; async function runTest () { let driver = new webdriver.Builder() .usingServer('https://api_key:api_secret@hub.testingbot.com/wd/hub') .withCapabilities(capabilities) .build(); await driver.get('extension://paaamahlcokbnafoiahanekijojkhbdm/popup/index.html'); const element = await driver.findElement(By.css('#environments')); const isElementDisplayed = await element.isDisplayed(); if (isElementDisplayed) { console.log('Element is present on the page.'); } else { console.log('Element is not present on the page.'); } await driver.quit(); } runTest(); ``` ## Safari Extension Testing Safari 16 and higher only TestingBot allows you to upload a zip file containing your (unsigned) Safari Web Extension or Safari App Extension. Simply provide a URL to a zip file containing the extension, or upload the zip file on [TestingBot Storage](https://testingbot.com#storage). You can now specify the URL to this file in the `load-extension` capability. When your test starts on a remote Safari instance, the extension will be enabled by default. TestingBot provides Safari Technology Preview as well, allowing you to test on the upcoming new Safari version. You can use the same extension testing features, simply change the `version` to `dev`. ### Safari Extension Automation TestingBot will automatically install and enable the extension in the Safari browser. Permissions will be granted automatically for the extension to access all webpages during the test. ![Safari permissions](https://testingbot.com/assets/support/web-automate/selenium/browser-extension/permissions-9541cd4aa43649ed64627925d68b08972d5267c39bea3823426fdb88ad7d6e49.png) ### Temporary Extension Installation [Since Safari 18.4](https://webkit.org/blog/16574/webkit-features-in-safari-18-4/#web-extensions) you can temporarily install web extensions, without having to build it with Xcode. You can simply zip your web extension and TestingBot will automatically install it as a temporary extension. You can try it on TestingBot's macOS Sequoia and macOS Tahoe machines. The zip file could contain for example following 4 files: `background.js`, `content.js`, `manifest.json` and `page.js`. ### UI Testing Safari Extensions To test the UI of your Safari extension, you will need to know the `safari-web-extension://` or `safari-app-extension://` URI. As Safari can change this to a random UUID, you will need to use the `tb:getExtensionId` command in your test to determine the correct URI to use. Safaridriver currently contains a bug that does not detect the load completion of a `safari-web-extension://` request. To work around this, set the [pageLoad](https://www.w3.org/TR/webdriver2/#timeouts) timeout to a low value and catch any timeout error. Please see the examples below on how to run a UI test against a Safari Web/App Extension: [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [C#](https://testingbot.com#) ```java import org.openqa.selenium.*; import org.openqa.selenium.remote.*; import java.net.URL; import java.util.*; public class SafariExtensionTest { public static void main(String[] args) throws Exception { MutableCapabilities caps = new MutableCapabilities(); caps.setCapability("browserName", "safari"); caps.setCapability("browserVersion", "18.0"); caps.setCapability("platformName", "SEQUOIA"); Map tbOptions = new HashMap<>(); tbOptions.put("load-extension", "tb://...safari-extension-path..."); caps.setCapability("tb:options", tbOptions); WebDriver driver = new RemoteWebDriver( new URL("https://API_KEY:API_SECRET@hub.testingbot.com/wd/hub"), caps ); driver.get("http://www.google.com/ncr"); ((JavascriptExecutor) driver).executeScript("tb:setPageLoadTimeout", 5); String extensionId = (String) ((JavascriptExecutor) driver).executeScript("tb:getExtensionId"); try { driver.get(extensionId + "popup/index.html"); } catch (TimeoutException e) { System.out.println(e.getMessage()); } System.out.println(driver.getPageSource()); driver.quit(); } } ``` ```python from selenium import webdriver from selenium.webdriver.safari.options import Options options = Options() options.set_capability("browserVersion", "18.0") options.set_capability("platformName", "SEQUOIA") options.set_capability("tb:options", { "load-extension": "tb://...safari-extension-path..." }) driver = webdriver.Remote( command_executor="https://API_KEY:API_SECRET@hub.testingbot.com/wd/hub", options=options ) driver.get("http://www.google.com/ncr") driver.set_page_load_timeout(5) extension_id = driver.execute_script("tb:getExtensionId") try: driver.get(f"{extension_id}popup/index.html") except Exception as e: print(e) print(driver.page_source) driver.quit() ``` ```javascript const { Builder } = require("selenium-webdriver"); (async () => { const caps = { browserName: "safari", browserVersion: "18.0", platformName: "SEQUOIA", "tb:options": { "load-extension": "tb://...safari-extension-path..." } }; const driver = await new Builder() .usingServer("https://API_KEY:API_SECRET@hub.testingbot.com/wd/hub") .withCapabilities(caps) .build(); await driver.get("http://www.google.com/ncr"); await driver.manage().setTimeouts({ pageLoad: 5000 }); const extensionId = await driver.executeScript("tb:getExtensionId"); try { await driver.get(`${extensionId}popup/index.html`); } catch (e) { console.log(e.message); } const pageSource = await driver.getPageSource(); console.log(pageSource); await driver.quit(); })(); ``` ```ruby require 'selenium-webdriver' options = Selenium::WebDriver::Safari::Options.new options.browser_version = '18.0' options.platform_name = 'SEQUOIA' options.add_option('tb:options', { 'load-extension': 'tb://...safari-extension-path...' }) http_client = Selenium::WebDriver::Remote::Http::Default.new http_client.read_timeout = 300 http_client.open_timeout = 300 driver = Selenium::WebDriver.for( :remote, url: "https://API_KEY:API_SECRET@hub.testingbot.com/wd/hub", options: options, http_client: http_client ) driver.navigate.to "http://www.google.com/ncr" driver.manage.timeouts.page_load = 5 extension_id = driver.execute_script("tb:getExtensionId") begin driver.navigate.to "#{extension_id}popup/index.html" rescue => e puts e.message end puts driver.page_source driver.quit ``` ```csharp using OpenQA.Selenium; using OpenQA.Selenium.Remote; using OpenQA.Selenium.Safari; using System; using System.Collections.Generic; class SafariExtensionTest { static void Main() { var options = new SafariOptions(); options.BrowserVersion = "18.0"; options.PlatformName = "SEQUOIA"; options.AddAdditionalOption("tb:options", new Dictionary { { "load-extension", "tb://...safari-extension-path..." } }); var driver = new RemoteWebDriver( new Uri("https://API_KEY:API_SECRET@hub.testingbot.com/wd/hub"), options ); driver.Navigate().GoToUrl("http://www.google.com/ncr"); driver.Manage().Timeouts().PageLoad = TimeSpan.FromSeconds(5); string extensionId = (string)((IJavaScriptExecutor)driver).ExecuteScript("tb:getExtensionId"); try { driver.Navigate().GoToUrl(extensionId + "popup/index.html"); } catch (WebDriverTimeoutException e) { Console.WriteLine(e.Message); } Console.WriteLine(driver.PageSource); driver.Quit(); } } ``` ### Older Safari versions For Safari versions up to Safari 12, you can use a `.safariextz` format, in a `zip` file, to run and test Safari extensions. ## iOS Mobile Safari Browser Extension Testing You can test your iOS Mobile Safari native extension with TestingBot. Simply upload your extension `.ipa` file to [TestingBot Storage](https://testingbot.com/support/api/storage#upload) and use it in your capabilities: ```javascript { "appium:app": "tb://path-to-mobile-safari-ext-ipa", "appium:includeSafariInWebviews": true, "appium:deviceName": "iPhone 16", "tb:options": { "realDevice": true } } ``` Next, you will need to allow this extension through the settings (preferences) app. Finally, we'll switch to the Safari app and change the context to Safari (WEBVIEW). Please see the examples below. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [C#](https://testingbot.com#) ```java import io.appium.java_client.AppiumBy; import io.appium.java_client.ios.IOSDriver; import io.appium.java_client.ios.options.XCUITestOptions; import org.openqa.selenium.WebElement; import java.net.URL; import java.util.Set; public class IOSAppiumAutomation { public static void main(String[] args) { try { XCUITestOptions options = new XCUITestOptions(); options.setApp("tb://path-to-mobile-safari-ext-ipa"); options.setDeviceName("iPhone 16"); options.setCapability("appium:includeSafariInWebviews", true); java.util.Map tbOptions = new java.util.HashMap<>(); tbOptions.put("realDevice", true); options.setCapability("tb:options", tbOptions); IOSDriver driver = new IOSDriver( new URL("https://hub.testingbot.com/wd/hub"), options); // 1. Open Safari Settings via deep link driver.executeScript("mobile: deepLink", java.util.Map.of("url", "app-prefs:com.apple.mobilesafari")); // 2. Tap "Extensions" WebElement extensionsCell = driver.findElement( AppiumBy.iOSNsPredicateString("type == 'XCUIElementTypeCell' AND name == 'Extensions'")); extensionsCell.click(); // 3. Tap your extension name WebElement extensionItem = driver.findElement( AppiumBy.iOSNsPredicateString("type == 'XCUIElementTypeCell' AND name == 'webrequest-har-collect'")); extensionItem.click(); // 4. Toggle "Allow Extension" WebElement allowSwitch = driver.findElement( AppiumBy.iOSNsPredicateString("type == 'XCUIElementTypeSwitch' AND name == 'Allow Extension'")); allowSwitch.click(); // 5. Tap "All Websites" WebElement allWebsites = driver.findElement( AppiumBy.iOSNsPredicateString("type == 'XCUIElementTypeCell' AND name == 'All Websites'")); allWebsites.click(); // 6. Tap "Allow" WebElement allow = driver.findElement( AppiumBy.iOSNsPredicateString("type == 'XCUIElementTypeCell' AND name == 'Allow'")); allow.click(); // 7. Terminate Settings driver.executeScript("mobile: terminateApp", java.util.Map.of("bundleId", "com.apple.Preferences")); // 8. Launch Safari driver.executeScript("mobile: launchApp", java.util.Map.of("bundleId", "com.apple.mobilesafari")); // 9. Switch to WebView context Set contexts = driver.getContextHandles(); for (String context : contexts) { if (context.contains("WEBVIEW")) { driver.context(context); break; } } System.out.println("Finished automation steps."); driver.quit(); } catch (Exception e) { e.printStackTrace(); } } } ``` ```python from appium import webdriver from appium.options.ios import XCUITestOptions from appium.webdriver.common.appiumby import AppiumBy options = XCUITestOptions() options.app = "tb://path-to-mobile-safari-ext-ipa" options.device_name = "iPhone 16" options.set_capability("appium:includeSafariInWebviews", True) options.set_capability("tb:options", {"realDevice": True}) driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options ) try: # 1. Open Safari settings via deep link driver.execute_script('mobile: deepLink', { 'url': 'app-prefs:com.apple.mobilesafari' }) # 2. Tap "Extensions" extensions_cell = driver.find_element( AppiumBy.IOS_PREDICATE, "type == 'XCUIElementTypeCell' AND name == 'Extensions'" ) extensions_cell.click() # 3. Tap on your extension (e.g., "webrequest-har-collect") extension_item = driver.find_element( AppiumBy.IOS_PREDICATE, "type == 'XCUIElementTypeCell' AND name == 'webrequest-har-collect'" ) extension_item.click() # 4. Enable the switch "Allow Extension" allow_switch = driver.find_element( AppiumBy.IOS_PREDICATE, "type == 'XCUIElementTypeSwitch' AND name == 'Allow Extension'" ) allow_switch.click() # 5. Tap "All Websites" all_websites = driver.find_element( AppiumBy.IOS_PREDICATE, "type == 'XCUIElementTypeCell' AND name == 'All Websites'" ) all_websites.click() # 6. Tap "Allow" allow = driver.find_element( AppiumBy.IOS_PREDICATE, "type == 'XCUIElementTypeCell' AND name == 'Allow'" ) allow.click() # 7. Terminate Settings app driver.execute_script('mobile: terminateApp', { 'bundleId': 'com.apple.Preferences' }) # 8. Launch Safari driver.execute_script('mobile: launchApp', { 'bundleId': 'com.apple.mobilesafari' }) # 9. Switch to WebView context contexts = driver.contexts webview = next((ctx for ctx in contexts if 'WEBVIEW' in ctx), None) if webview: driver.switch_to.context(webview) print("Finished automation steps.") finally: driver.quit() ``` ```javascript const { remote } = require('webdriverio'); (async () => { const driver = await remote({ path: 'https://hub.testingbot.com/wd/hub', port: 443, capabilities: { 'appium:app': 'tb://path-to-mobile-safari-ext-ipa', platformName: 'iOS', 'appium:automationName': 'XCUITest', 'appium:includeSafariInWebviews': true, 'appium:deviceName': 'iPhone 16', 'tb:options': { realDevice: true } }, }); // 1. Open Safari settings via deep link await driver.execute('mobile: deepLink', { url: 'app-prefs:com.apple.mobilesafari', }); // 2. Tap "Extensions" const extensionsCell = await driver.$(`-ios predicate string:type == 'XCUIElementTypeCell' AND name == 'Extensions'`); await extensionsCell.waitForExist({ timeout: 10000 }); await extensionsCell.click(); // 3. Tap on your extension (e.g., "webrequest-har-collect") const extensionItem = await driver.$(`-ios predicate string:type == 'XCUIElementTypeCell' AND name == 'webrequest-har-collect'`); await extensionItem.waitForExist({ timeout: 60000 }); await extensionItem.click(); // 4. Enable the switch "Allow Extension" const allowSwitch = await driver.$(`-ios predicate string:type == 'XCUIElementTypeSwitch' AND name == 'Allow Extension'`); await allowSwitch.waitForExist({ timeout: 60000 }); await allowSwitch.click(); // 5. Tap "All Websites" const allWebsites = await driver.$(`-ios predicate string:type == 'XCUIElementTypeCell' AND name == 'All Websites'`); await allWebsites.waitForExist({ timeout: 60000 }); await allWebsites.click(); // 6. Tap "Allow" const allow = await driver.$(`-ios predicate string:type == 'XCUIElementTypeCell' AND name == 'Allow'`); await allow.waitForExist({ timeout: 60000 }); await allow.click(); // 7. Terminate Settings app await driver.execute('mobile: terminateApp', { bundleId: 'com.apple.Preferences', }); // 8. Launch Safari await driver.execute('mobile: launchApp', { bundleId: 'com.apple.mobilesafari', }); // 9. Switch to WebView context const contexts = await driver.getContexts(); const webview = contexts.find(ctx => ctx.includes('WEBVIEW')); if (webview) { await driver.switchContext(webview); } console.log('Finished automation steps.'); await driver.deleteSession(); })(); ``` ```ruby require 'appium_lib' caps = { caps: { platformName: 'iOS', 'appium:automationName': 'XCUITest', 'appium:app': 'tb://path-to-mobile-safari-ext-ipa', 'appium:deviceName': 'iPhone 16', 'appium:includeSafariInWebviews': true, 'tb:options': { 'realDevice' => true } }, appium_lib: { server_url: 'https://hub.testingbot.com/wd/hub' } } driver = Appium::Driver.new(caps, true) driver.start_driver begin # 1. Open Safari settings via deep link driver.execute_script('mobile: deepLink', { url: 'app-prefs:com.apple.mobilesafari' }) # 2. Tap "Extensions" extensions = driver.find_element(:predicate, "type == 'XCUIElementTypeCell' AND name == 'Extensions'") extensions.click # 3. Tap your extension extension = driver.find_element(:predicate, "type == 'XCUIElementTypeCell' AND name == 'webrequest-har-collect'") extension.click # 4. Enable "Allow Extension" allow_switch = driver.find_element(:predicate, "type == 'XCUIElementTypeSwitch' AND name == 'Allow Extension'") allow_switch.click # 5. Tap "All Websites" all_sites = driver.find_element(:predicate, "type == 'XCUIElementTypeCell' AND name == 'All Websites'") all_sites.click # 6. Tap "Allow" allow = driver.find_element(:predicate, "type == 'XCUIElementTypeCell' AND name == 'Allow'") allow.click # 7. Terminate Settings app driver.execute_script('mobile: terminateApp', { bundleId: 'com.apple.Preferences' }) # 8. Launch Safari driver.execute_script('mobile: launchApp', { bundleId: 'com.apple.mobilesafari' }) # 9. Switch to WebView context contexts = driver.available_contexts webview = contexts.find { |ctx| ctx.include?('WEBVIEW') } driver.set_context(webview) if webview puts 'Finished automation steps.' ensure driver.quit_driver end ``` ```csharp using OpenQA.Selenium; using OpenQA.Selenium.Appium; using OpenQA.Selenium.Appium.iOS; using System; using System.Collections.Generic; using System.Collections.ObjectModel; using System.Linq; class Program { static void Main(string[] args) { var options = new AppiumOptions(); options.PlatformName = "iOS"; options.AutomationName = "XCUITest"; options.AddAdditionalAppiumOption("app", "tb://path-to-mobile-safari-ext-ipa"); options.AddAdditionalAppiumOption("deviceName", "iPhone 16"); options.AddAdditionalAppiumOption("includeSafariInWebviews", true); options.AddAdditionalOption("tb:options", new Dictionary { ["realDevice"] = true }); var driver = new IOSDriver( new Uri("https://hub.testingbot.com/wd/hub"), options ); try { // 1. Open Safari settings via deep link driver.ExecuteScript("mobile: deepLink", new Dictionary { { "url", "app-prefs:com.apple.mobilesafari" } }); // 2. Tap "Extensions" var extensionsCell = driver.FindElement(MobileBy.IosNsPredicateString( "type == 'XCUIElementTypeCell' AND name == 'Extensions'")); extensionsCell.Click(); // 3. Tap extension (e.g., "webrequest-har-collect") var extensionItem = driver.FindElement(MobileBy.IosNsPredicateString( "type == 'XCUIElementTypeCell' AND name == 'webrequest-har-collect'")); extensionItem.Click(); // 4. Tap "Allow Extension" switch var allowSwitch = driver.FindElement(MobileBy.IosNsPredicateString( "type == 'XCUIElementTypeSwitch' AND name == 'Allow Extension'")); allowSwitch.Click(); // 5. Tap "All Websites" var allWebsites = driver.FindElement(MobileBy.IosNsPredicateString( "type == 'XCUIElementTypeCell' AND name == 'All Websites'")); allWebsites.Click(); // 6. Tap "Allow" var allow = driver.FindElement(MobileBy.IosNsPredicateString( "type == 'XCUIElementTypeCell' AND name == 'Allow'")); allow.Click(); // 7. Terminate Settings app driver.ExecuteScript("mobile: terminateApp", new Dictionary { { "bundleId", "com.apple.Preferences" } }); // 8. Launch Safari driver.ExecuteScript("mobile: launchApp", new Dictionary { { "bundleId", "com.apple.mobilesafari" } }); // 9. Switch to WebView context ReadOnlyCollection contexts = driver.Contexts; string webview = contexts.FirstOrDefault(c => c.Contains("WEBVIEW")); if (webview != null) { driver.Context = webview; } Console.WriteLine("Finished automation steps."); } finally { driver.Quit(); } } } ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/firefox-profile # Firefox Profile With Selenium WebDriver, it's possible to set a custom [Firefox profile](https://support.mozilla.org/en-US/kb/profile-manager-create-remove-switch-firefox-profiles), which will be used during your automated tests on TestingBot. To change preferences and settings within Firefox, you'll need to instantiate a new `FirefoxProfile` object and update the properties of the profile. ## Change Firefox Profile In the example below, we'll set the `accept language` to Spanish and change the `user-agent` header. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java FirefoxOptions options = new FirefoxOptions(); FirefoxProfile profile = new FirefoxProfile(); profile.setPreference("general.useragent.override", "Custom-Agent"); // Change user-agent profile.setPreference("intl.accept_languages", "es"); // Setting accept language to Spanish options.setProfile(profile); options.setPlatformName("WIN11"); options.setBrowserVersion("latest"); HashMap tbOptions = new HashMap<>(); tbOptions.put("key", "API_KEY"); tbOptions.put("secret", "API_SECRET"); tbOptions.put("name", "firefox profile example"); options.setCapability("tb:options", tbOptions); WebDriver driver = new RemoteWebDriver(new URL("https://hub.testingbot.com/wd/hub"), options); ``` ```python from selenium import webdriver from selenium.webdriver.firefox.options import Options options = Options() options.set_preference('general.useragent.override', 'Custom-Agent') # Change user-agent options.set_preference('intl.accept_languages', 'es') # Setting accept language to Spanish options.browser_version = 'latest' options.platform_name = 'WIN11' options.set_capability('tb:options', { 'key': 'API_KEY', 'secret': 'API_SECRET', 'name': 'firefox profile example' }) driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options ) ``` ```javascript const { Builder } = require('selenium-webdriver'); const firefox = require('selenium-webdriver/firefox'); async function main() { let options = new firefox.Options(); options.setPreference('general.useragent.override', 'Custom-Agent'); // Change user-agent options.setPreference('intl.accept_languages', 'es'); // Setting accept language to Spanish options.setBrowserVersion('latest'); options.setPlatformName('WIN11'); options.set('tb:options', { 'key': 'API_KEY', 'secret': 'API_SECRET', 'name': 'firefox profile example' }); let driver = await new Builder() .usingServer('https://hub.testingbot.com/wd/hub') .setFirefoxOptions(options) .build(); } main(); ``` ```ruby require 'rubygems' require 'selenium-webdriver' profile = Selenium::WebDriver::Firefox::Profile.new profile['general.useragent.override'] = 'Custom-Agent' # Change user-agent profile['intl.accept_languages'] = 'es' # Setting accept language to Spanish options = Selenium::WebDriver::Firefox::Options.new options.profile = profile options.browser_version = 'latest' options.platform_name = 'WIN11' options.add_option('tb:options', { 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'firefox profile example' }) driver = Selenium::WebDriver.for(:remote, url: 'https://hub.testingbot.com/wd/hub', options: options) ``` ```php require_once('vendor/autoload.php'); use Facebook\WebDriver\Remote\RemoteWebDriver; use Facebook\WebDriver\Remote\DesiredCapabilities; use Facebook\WebDriver\Firefox\FirefoxOptions; use Facebook\WebDriver\Firefox\FirefoxProfile; $profile = new FirefoxProfile(); $profile->setPreference('general.useragent.override', 'Custom-Agent'); // Change user-agent $profile->setPreference('intl.accept_languages', 'es'); // Setting accept language to Spanish $options = new FirefoxOptions(); $options->setProfile($profile); $capabilities = DesiredCapabilities::firefox(); $capabilities->setCapability(FirefoxOptions::CAPABILITY, $options); $capabilities->setCapability('browserVersion', 'latest'); $capabilities->setCapability('platformName', 'WIN11'); $capabilities->setCapability('tb:options', [ 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'firefox profile example' ]); $web_driver = RemoteWebDriver::create( 'https://hub.testingbot.com/wd/hub', $capabilities ); ``` ```csharp FirefoxProfile profile = new FirefoxProfile(); profile.SetPreference("general.useragent.override", "Custom-Agent"); // Change user-agent profile.SetPreference("intl.accept_languages", "es"); // Setting accept language to Spanish FirefoxOptions options = new FirefoxOptions(); options.Profile = profile; options.BrowserVersion = "latest"; options.PlatformName = "WIN11"; options.AddAdditionalOption("tb:options", new Dictionary { ["key"] = "API_KEY", ["secret"] = "API_SECRET", ["name"] = "firefox profile example" }); IWebDriver driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), options ); ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/migrate-existing-tests # Migrate your existing tests Migrating your existing Selenium WebDriver tests to use TestingBot is easy. On this page, we'll show you how to modify your existing tests and run them on the browsers in our cloud. ## Authentication First, you'll need to authenticate to the TestingBot cloud. If you haven't already, please [sign up](https://testingbot.com/users/sign_up) for a free trial and find your TestingBot key and secret in the member area. You'll need to use these credentials in your test scripts to authenticate with our cloud. ## Capabilities Capabilities are used to indicate on which browsers you want to test. You can specify a desired browser name, operating system and browser version. For example, let's say you want to test on the latest Chrome browser on Windows 10: [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby options = Selenium::WebDriver::Chrome::Options.new options.browser_version = 'latest' options.platform_name = 'WIN11' options.add_option('tb:options', { 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'My First Test' }) ``` ```java ChromeOptions options = new ChromeOptions(); options.setPlatformName("WIN11"); options.setBrowserVersion("latest"); HashMap tbOptions = new HashMap<>(); tbOptions.put("key", "API_KEY"); tbOptions.put("secret", "API_SECRET"); tbOptions.put("name", "My First Test"); options.setCapability("tb:options", tbOptions); ``` ```php $capabilities = DesiredCapabilities::chrome(); $capabilities->setCapability('platformName', 'WIN11'); $capabilities->setCapability('browserVersion', 'latest'); $capabilities->setCapability('tb:options', [ 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'My First Test' ]); ``` ```python from selenium.webdriver.chrome.options import Options options = Options() options.browser_version = 'latest' options.platform_name = 'WIN11' options.set_capability('tb:options', { 'key': 'API_KEY', 'secret': 'API_SECRET', 'name': 'My First Test' }) ``` ```javascript const chrome = require('selenium-webdriver/chrome'); let options = new chrome.Options(); options.setBrowserVersion('latest'); options.setPlatformName('WIN11'); options.set('tb:options', { 'key': 'API_KEY', 'secret': 'API_SECRET', 'name': 'My First Test' }); ``` ```csharp // for Chrome ChromeOptions options = new ChromeOptions(); // for Firefox // FirefoxOptions options = new FirefoxOptions(); // for Safari // SafariOptions options = new SafariOptions(); options.PlatformName = "WIN11"; options.BrowserVersion = "latest"; options.AddAdditionalOption("tb:options", new Dictionary { ["key"] = "API_KEY", ["secret"] = "API_SECRET", ["name"] = "My First Test" }); ``` ## Running tests with Remote WebDriver If you have been running tests on your local machine until now, then you will need to modify your test scripts to start a WebDriver session via Remote WebDriver. Simply send the [capabilities](https://testingbot.com#capabilities) to our cloud endpoint: ```bash https://hub.testingbot.com/wd/hub ``` Below is an example on how to do this: [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby require 'selenium-webdriver' options = Selenium::WebDriver::Chrome::Options.new options.browser_version = 'latest' options.platform_name = 'WIN11' options.add_option('tb:options', { 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'My First Test' }) driver = Selenium::WebDriver.for(:remote, url: 'https://hub.testingbot.com/wd/hub', options: options ) driver.navigate.to 'https://www.google.com' # Your test logic here driver.quit ``` ```java import org.openqa.selenium.WebDriver; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URL; import java.util.HashMap; public class TestingBotSampleTest { public static void main(String[] args) throws Exception { ChromeOptions options = new ChromeOptions(); options.setPlatformName("WIN11"); options.setBrowserVersion("latest"); HashMap tbOptions = new HashMap<>(); tbOptions.put("key", "API_KEY"); tbOptions.put("secret", "API_SECRET"); tbOptions.put("name", "My First Test"); options.setCapability("tb:options", tbOptions); WebDriver driver = new RemoteWebDriver( new URL("https://hub.testingbot.com/wd/hub"), options ); driver.get("https://www.google.com"); // Your test logic here driver.quit(); } } ``` ```php setCapability('platformName', 'WIN11'); $capabilities->setCapability('browserVersion', 'latest'); $capabilities->setCapability('tb:options', [ 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'My First Test' ]); $driver = RemoteWebDriver::create( 'https://hub.testingbot.com/wd/hub', $capabilities ); $driver->get('https://www.google.com'); // Your test logic here $driver->quit(); ``` ```python from selenium import webdriver from selenium.webdriver.chrome.options import Options options = Options() options.browser_version = 'latest' options.platform_name = 'WIN11' options.set_capability('tb:options', { 'key': 'API_KEY', 'secret': 'API_SECRET', 'name': 'My First Test' }) driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options ) driver.get('https://www.google.com') # Your test logic here driver.quit() ``` ```javascript const { Builder } = require('selenium-webdriver'); const chrome = require('selenium-webdriver/chrome'); async function runTest() { let options = new chrome.Options(); options.setBrowserVersion('latest'); options.setPlatformName('WIN11'); options.set('tb:options', { 'key': 'API_KEY', 'secret': 'API_SECRET', 'name': 'My First Test' }); let driver = await new Builder() .usingServer('https://hub.testingbot.com/wd/hub') .setChromeOptions(options) .build(); await driver.get('https://www.google.com'); // Your test logic here await driver.quit(); } runTest(); ``` ```csharp using System; using System.Collections.Generic; using OpenQA.Selenium; using OpenQA.Selenium.Remote; using OpenQA.Selenium.Chrome; namespace SeleniumTest { class Program { static void Main(string[] args) { ChromeOptions options = new ChromeOptions(); options.PlatformName = "WIN11"; options.BrowserVersion = "latest"; options.AddAdditionalOption("tb:options", new Dictionary { ["key"] = "API_KEY", ["secret"] = "API_SECRET", ["name"] = "My First Test" }); IWebDriver driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), options ); driver.Navigate().GoToUrl("https://www.google.com"); // Your test logic here driver.Quit(); } } } ``` ## Migrate from one of the test frameworks We have documentation for all the [popular test frameworks](https://testingbot.com/support/web-automate/selenium). Simply follow the documentation and examples to convert your existing tests. If the test framework you are using is not in our documentation, please [contact us](https://testingbot.com/contact/new) and we'll be happy to help. ## Test your private websites on TestingBot TestingBot offers a feature called [TestingBot Tunnel](https://testingbot.com/support/tunnel) which allows you to run tests against websites hosted on your computer or private network. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/permission-popups # Handling permission pop-ups The website you are testing may ask for specific permissions during testing. For example: - [Permission to access the device location](https://testingbot.com#location) - [Permission to access the camera and/or microphone](https://testingbot.com#camera) - [Permission to show notifications](https://testingbot.com#notifications) - [Permission to access clipboard](https://testingbot.com#clipboard) The browser might show a popup or prompt, asking for user permission. In case of Automated Testing, you'll need a way to automatically approve/deny these requests. Below we'll show you how to do this with your Automated Tests. ## Location Permission A popup will appear when the website or mobile app asks the user for the location. You can choose to **Allow** or **Block** this request. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java import org.openqa.selenium.By; import org.openqa.selenium.WebDriver; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URL; import java.util.HashMap; import java.util.Map; public class AllowLocationPopup { public static final String HUB_URL = "https://hub.testingbot.com/wd/hub"; public static void main(String[] args) throws Exception { ChromeOptions options = new ChromeOptions(); options.setPlatformName("WIN11"); options.setBrowserVersion("latest"); Map prefs = new HashMap<>(); Map profile = new HashMap<>(); Map contentSettings = new HashMap<>(); // 0 - Default, 1 - Allow, 2 - Block contentSettings.put("geolocation", 1); profile.put("default_content_setting_values", contentSettings); prefs.put("profile", profile); options.setExperimentalOption("prefs", prefs); Map tbOptions = new HashMap<>(); tbOptions.put("key", "API_KEY"); tbOptions.put("secret", "API_SECRET"); options.setCapability("tb:options", tbOptions); WebDriver driver = new RemoteWebDriver(new URL(HUB_URL), options); driver.get("https://the-internet.herokuapp.com/geolocation"); driver.findElement(By.xpath("//*[@id='content']/div/button")).click(); Thread.sleep(5000); driver.quit(); } } ``` ```python from time import sleep from selenium import webdriver from selenium.webdriver.chrome.options import Options chrome_options = Options() # 0 - Default, 1 - Allow, 2 - Block chrome_options.add_experimental_option("prefs", { "profile.default_content_setting_values.geolocation": 1}) chrome_options.platform_name = "WIN11" chrome_options.browser_version = "latest" chrome_options.set_capability("tb:options", { "key": "API_KEY", "secret": "API_SECRET" }) driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=chrome_options ) driver.get("https://the-internet.herokuapp.com/geolocation") driver.find_element("xpath", "//*[@id='content']/div/button").click() sleep(10) driver.quit() ``` ```javascript const { Builder, By } = require('selenium-webdriver'); const capabilities = { 'browserName': 'chrome', 'browserVersion': 'latest', 'platformName': 'WIN11', 'tb:options': { 'key': 'API_KEY', 'secret': 'API_SECRET' }, 'goog:chromeOptions': { prefs: { // 0 - Default, 1 - Allow, 2 - Block 'profile.default_content_setting_values.geolocation': 1 } } }; (async function example() { const driver = await new Builder() .usingServer('https://hub.testingbot.com/wd/hub') .withCapabilities(capabilities) .build(); await driver.get('https://the-internet.herokuapp.com/geolocation'); await driver.findElement(By.xpath("//*[@id='content']/div/button")).click(); await driver.sleep(5000); await driver.quit(); })(); ``` ```ruby require 'selenium-webdriver' options = Selenium::WebDriver::Chrome::Options.new options.add_preference("profile.default_content_setting_values.geolocation", 1) options.add_option("platformName", "WIN11") options.add_option("browserVersion", "latest") options.add_option("tb:options", { key: "API_KEY", secret: "API_SECRET" }) driver = Selenium::WebDriver.for :remote, url: "https://hub.testingbot.com/wd/hub", options: options driver.navigate.to "https://the-internet.herokuapp.com/geolocation" driver.find_element(:xpath, "//*[@id='content']/div/button").click sleep 5 driver.quit ``` ```php setExperimentalOption('prefs', [ 'profile.default_content_setting_values.geolocation' => 1 ]); $caps = DesiredCapabilities::chrome(); $caps->setCapability(ChromeOptions::CAPABILITY, $options); $caps->setCapability('platformName', 'WIN11'); $caps->setCapability('browserVersion', 'latest'); $caps->setCapability('tb:options', [ 'key' => 'API_KEY', 'secret' => 'API_SECRET' ]); $driver = RemoteWebDriver::create( "https://hub.testingbot.com/wd/hub", $caps, 120000 ); $driver->get("https://the-internet.herokuapp.com/geolocation"); $driver->findElement(WebDriverBy::xpath("//*[@id='content']/div/button"))->click(); sleep(5); $driver->quit(); ?> ``` ```csharp using System; using System.Collections.Generic; using System.Threading; using OpenQA.Selenium; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Remote; namespace GeoCSharp { public class Program { public static void Main(string[] args) { ChromeOptions options = new ChromeOptions(); options.PlatformName = "WIN11"; options.BrowserVersion = "latest"; // 0 - Default, 1 - Allow, 2 - Block options.AddUserProfilePreference("profile.default_content_setting_values.geolocation", 1); var tbOptions = new Dictionary { ["key"] = "API_KEY", ["secret"] = "API_SECRET" }; options.AddAdditionalOption("tb:options", tbOptions); IWebDriver driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), options); driver.Navigate().GoToUrl("https://the-internet.herokuapp.com/geolocation"); driver.FindElement(By.XPath("//*[@id='content']/div/button")).Click(); Thread.Sleep(5000); driver.Quit(); } } } ``` ## Camera and Microphone Permissions During your Automated Testing, you might want to test and handle Camera and Microphone permissions. Please see the example below, it shows you how to either **Allow** or **Block** a request from your webapp. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java import org.openqa.selenium.By; import org.openqa.selenium.WebDriver; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URL; import java.util.HashMap; import java.util.Map; public class AllowCameraPopupChrome { public static final String HUB_URL = "https://hub.testingbot.com/wd/hub"; public static void main(String[] args) throws Exception { // Configure ChromeOptions to pass a fake media stream ChromeOptions options = new ChromeOptions(); options.addArguments("use-fake-device-for-media-stream"); options.addArguments("use-fake-ui-for-media-stream"); options.setPlatformName("WIN11"); options.setBrowserVersion("latest"); Map tbOptions = new HashMap<>(); tbOptions.put("key", "API_KEY"); tbOptions.put("secret", "API_SECRET"); options.setCapability("tb:options", tbOptions); WebDriver driver = new RemoteWebDriver(new URL(HUB_URL), options); // WebCam Test driver.get("https://webcamtests.com/check"); Thread.sleep(5000); driver.findElement(By.id("webcam-launcher")).click(); Thread.sleep(2000); // Microphone Test driver.get("https://www.vidyard.com/mic-test/"); Thread.sleep(2000); driver.findElement(By.xpath("//a[@id='start-test']")).click(); Thread.sleep(2000); driver.quit(); } } ``` ```python from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.chrome.options import Options import time # Configure ChromeOptions to pass fake media stream chrome_options = Options() chrome_options.add_argument("--use-fake-device-for-media-stream") chrome_options.add_argument("--use-fake-ui-for-media-stream") chrome_options.platform_name = "WIN11" chrome_options.browser_version = "latest" chrome_options.set_capability("tb:options", { "key": "API_KEY", "secret": "API_SECRET" }) driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=chrome_options ) # WebCam Test driver.get("https://webcamtests.com/check") time.sleep(5) driver.find_element(By.ID, "webcam-launcher").click() time.sleep(2) # Mic Test driver.get("https://www.vidyard.com/mic-test/") time.sleep(5) driver.find_element(By.XPATH, "//a[@id='start-test']").click() time.sleep(2) driver.quit() ``` ```javascript const { Builder, By } = require('selenium-webdriver'); const capabilities = { 'browserName': 'chrome', 'browserVersion': 'latest', 'platformName': 'WIN11', 'tb:options': { 'key': 'API_KEY', 'secret': 'API_SECRET' }, // Configure ChromeOptions to pass fake media stream 'goog:chromeOptions': { 'args': ['--use-fake-device-for-media-stream', '--use-fake-ui-for-media-stream'] } }; (async function example() { const driver = await new Builder() .usingServer('https://hub.testingbot.com/wd/hub') .withCapabilities(capabilities) .build(); // Webcam Test await driver.get('https://webcamtests.com/check'); await driver.sleep(5000); await driver.findElement(By.id('webcam-launcher')).click(); await driver.sleep(2000); // Microphone Test await driver.get('https://www.vidyard.com/mic-test/'); await driver.sleep(2000); await driver.findElement(By.xpath("//a[@id='start-test']")).click(); await driver.sleep(2000); await driver.quit(); })(); ``` ```ruby require 'selenium-webdriver' # Configure ChromeOptions to pass a fake media stream during testing options = Selenium::WebDriver::Chrome::Options.new options.add_argument('--use-fake-device-for-media-stream') options.add_argument('--use-fake-ui-for-media-stream') options.add_option("platformName", "WIN11") options.add_option("browserVersion", "latest") options.add_option("tb:options", { key: "API_KEY", secret: "API_SECRET" }) driver = Selenium::WebDriver.for :remote, url: "https://hub.testingbot.com/wd/hub", options: options # Web Cam Test driver.navigate.to "https://webcamtests.com/check" sleep(5) driver.find_element(:id, "webcam-launcher").click sleep(2) # Microphone Test driver.navigate.to "https://www.vidyard.com/mic-test/" sleep(5) driver.find_element(:xpath, "//a[@id='start-test']").click sleep(2) driver.quit ``` ```php addArguments(['--use-fake-device-for-media-stream', '--use-fake-ui-for-media-stream']); $caps = DesiredCapabilities::chrome(); $caps->setCapability(ChromeOptions::CAPABILITY, $options); $caps->setCapability('platformName', 'WIN11'); $caps->setCapability('browserVersion', 'latest'); $caps->setCapability('tb:options', [ 'key' => 'API_KEY', 'secret' => 'API_SECRET' ]); $driver = RemoteWebDriver::create( "https://hub.testingbot.com/wd/hub", $caps, 120000 ); // WebCam Test $driver->get("https://webcamtests.com/check"); sleep(5); $driver->findElement(WebDriverBy::id("webcam-launcher"))->click(); sleep(2); // Microphone Test $driver->get("https://www.vidyard.com/mic-test/"); sleep(5); $driver->findElement(WebDriverBy::xpath("//a[@id='start-test']"))->click(); sleep(2); $driver->quit(); ?> ``` ```csharp using System; using System.Collections.Generic; using System.Threading; using OpenQA.Selenium; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Remote; namespace SampleCSharp { public class AllowCameraMicPopupChrome { static void Main(string[] args) { // Configure ChromeOptions to pass fake media stream ChromeOptions options = new ChromeOptions(); options.AddArgument("--use-fake-device-for-media-stream"); options.AddArgument("--use-fake-ui-for-media-stream"); options.PlatformName = "WIN11"; options.BrowserVersion = "latest"; var tbOptions = new Dictionary { ["key"] = "API_KEY", ["secret"] = "API_SECRET" }; options.AddAdditionalOption("tb:options", tbOptions); IWebDriver driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), options); // Webcam Test driver.Navigate().GoToUrl("https://webcamtests.com/check"); Thread.Sleep(5000); driver.FindElement(By.Id("webcam-launcher")).Click(); Thread.Sleep(2000); // Microphone Test driver.Navigate().GoToUrl("https://www.vidyard.com/mic-test/"); Thread.Sleep(2000); driver.FindElement(By.XPath("//a[@id='start-test']")).Click(); Thread.Sleep(2000); driver.Quit(); } } } ``` ## Notification Permissions During your Automated Testing, the website might request access to show notifications. Please see the example below, it shows you how to either **Allow** or **Block** a notification request. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java import org.openqa.selenium.By; import org.openqa.selenium.WebDriver; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URL; import java.util.HashMap; import java.util.Map; public class AllowNotificationPopupDesktop { public static final String HUB_URL = "https://hub.testingbot.com/wd/hub"; public static void main(String[] args) throws Exception { ChromeOptions options = new ChromeOptions(); options.setPlatformName("WIN11"); options.setBrowserVersion("latest"); Map prefs = new HashMap<>(); Map profile = new HashMap<>(); Map contentSettings = new HashMap<>(); // 0 - Default, 1 - Allow, 2 - Block contentSettings.put("notifications", 1); profile.put("default_content_setting_values", contentSettings); prefs.put("profile", profile); options.setExperimentalOption("prefs", prefs); Map tbOptions = new HashMap<>(); tbOptions.put("key", "API_KEY"); tbOptions.put("secret", "API_SECRET"); options.setCapability("tb:options", tbOptions); WebDriver driver = new RemoteWebDriver(new URL(HUB_URL), options); driver.get("https://web-push-book.gauntface.com/demos/notification-examples/"); driver.findElement(By.xpath("//body/main[1]/p[3]/input[1]")).click(); Thread.sleep(2000); driver.quit(); } } ``` ```python from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.chrome.options import Options import time # 0 - Default, 1 - Allow, 2 - Block chrome_options = Options() chrome_options.add_experimental_option("prefs", { "profile.default_content_setting_values.notifications": 1 }) chrome_options.platform_name = "WIN11" chrome_options.browser_version = "latest" chrome_options.set_capability("tb:options", { "key": "API_KEY", "secret": "API_SECRET" }) driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=chrome_options ) driver.get("https://web-push-book.gauntface.com/demos/notification-examples/") driver.find_element(By.XPATH, "//body/main[1]/p[3]/input[1]").click() time.sleep(2) driver.quit() ``` ```javascript const { Builder, By } = require('selenium-webdriver'); const capabilities = { 'browserName': 'chrome', 'browserVersion': 'latest', 'platformName': 'WIN11', 'tb:options': { 'key': 'API_KEY', 'secret': 'API_SECRET' }, 'goog:chromeOptions': { prefs: { // 0 - Default, 1 - Allow, 2 - Block 'profile.default_content_setting_values.notifications': 1 } } }; (async function example() { const driver = await new Builder() .usingServer('https://hub.testingbot.com/wd/hub') .withCapabilities(capabilities) .build(); await driver.get('https://web-push-book.gauntface.com/demos/notification-examples/'); await driver.findElement(By.xpath("//body/main[1]/p[3]/input[1]")).click(); await driver.sleep(2000); await driver.quit(); })(); ``` ```ruby require 'selenium-webdriver' # 0 - Default, 1 - Allow, 2 - Block options = Selenium::WebDriver::Chrome::Options.new options.add_preference("profile.default_content_setting_values.notifications", 1) options.add_option("platformName", "WIN11") options.add_option("browserVersion", "latest") options.add_option("tb:options", { key: "API_KEY", secret: "API_SECRET" }) driver = Selenium::WebDriver.for :remote, url: "https://hub.testingbot.com/wd/hub", options: options driver.navigate.to "https://web-push-book.gauntface.com/demos/notification-examples/" driver.find_element(:xpath, "//body/main[1]/p[3]/input[1]").click sleep(2) driver.quit ``` ```php setExperimentalOption('prefs', [ 'profile.default_content_setting_values.notifications' => 1 ]); $caps = DesiredCapabilities::chrome(); $caps->setCapability(ChromeOptions::CAPABILITY, $options); $caps->setCapability('platformName', 'WIN11'); $caps->setCapability('browserVersion', 'latest'); $caps->setCapability('tb:options', [ 'key' => 'API_KEY', 'secret' => 'API_SECRET' ]); $driver = RemoteWebDriver::create( "https://hub.testingbot.com/wd/hub", $caps, 120000 ); $driver->get("https://web-push-book.gauntface.com/demos/notification-examples/"); $driver->findElement(WebDriverBy::xpath("//body/main[1]/p[3]/input[1]"))->click(); sleep(2); $driver->quit(); ?> ``` ```csharp using System; using System.Collections.Generic; using System.Threading; using OpenQA.Selenium; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Remote; namespace SampleCSharp { public class Program { static void Main(string[] args) { ChromeOptions options = new ChromeOptions(); options.PlatformName = "WIN11"; options.BrowserVersion = "latest"; // 0 - Default, 1 - Allow, 2 - Block options.AddUserProfilePreference("profile.default_content_setting_values.notifications", 1); var tbOptions = new Dictionary { ["key"] = "API_KEY", ["secret"] = "API_SECRET" }; options.AddAdditionalOption("tb:options", tbOptions); IWebDriver driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), options); driver.Navigate().GoToUrl("https://web-push-book.gauntface.com/demos/notification-examples/"); driver.FindElement(By.XPath("//body/main[1]/p[3]/input[1]")).Click(); Thread.Sleep(2000); driver.Quit(); } } } ``` ## Clipboard Permissions During your Automated Testing, Javascript code on the website you are testing might ask for clipboard permission. This permission dialog asks the user if the website can read (and/or write) to the clipboard. ![Clipboard Access Request example](https://testingbot.com/assets/support/clipboard-permission-d786d434b15e680ce09b16d2655082be94f712a638ccba2394051f7cdc6cc962.webp) Usually, this is accomplished by running a piece of Javascript code, similar to this snippet: ```javascript const text = await navigator.clipboard.readText(); ``` Please see the example below, it shows you how to either **Allow** or **Block** a clipboard access request. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java import org.openqa.selenium.By; import org.openqa.selenium.WebDriver; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URL; import java.util.HashMap; import java.util.Map; public class AllowClipboardPermission { public static final String HUB_URL = "https://hub.testingbot.com/wd/hub"; private static Map getClipBoardSettingsMap(int settingValue) { Map map = new HashMap<>(); map.put("last_modified", String.valueOf(System.currentTimeMillis())); map.put("setting", settingValue); Map cbPreference = new HashMap<>(); cbPreference.put("[*.],*", map); return cbPreference; } public static void main(String[] args) throws Exception { ChromeOptions options = new ChromeOptions(); options.setPlatformName("WIN11"); options.setBrowserVersion("latest"); Map prefs = new HashMap<>(); prefs.put("profile.content_settings.exceptions.clipboard", getClipBoardSettingsMap(1)); options.setExperimentalOption("prefs", prefs); Map tbOptions = new HashMap<>(); tbOptions.put("key", "API_KEY"); tbOptions.put("secret", "API_SECRET"); options.setCapability("tb:options", tbOptions); WebDriver driver = new RemoteWebDriver(new URL(HUB_URL), options); driver.get("https://permission.site/#read-text"); driver.findElement(By.id("read-text")).click(); Thread.sleep(2000); driver.quit(); } } ``` ```python from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.chrome.options import Options import time chrome_options = Options() chrome_options.add_experimental_option('prefs', { 'profile.content_settings.exceptions.clipboard': { '*': {'setting': 1} } }) chrome_options.platform_name = "WIN11" chrome_options.browser_version = "latest" chrome_options.set_capability("tb:options", { "key": "API_KEY", "secret": "API_SECRET" }) driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=chrome_options ) driver.get("https://permission.site/#read-text") driver.find_element(By.ID, "read-text").click() time.sleep(2) driver.quit() ``` ```javascript const { Builder, By } = require('selenium-webdriver'); const chrome = require('selenium-webdriver/chrome'); const chromeOptions = new chrome.Options(); chromeOptions.setUserPreferences({ profile: { content_settings: { exceptions: { clipboard: { 'https://permission.site,*': { expiration: '0', last_modified: Date.now().toString(), model: 0, setting: 1 } } } } } }); const capabilities = { 'browserName': 'chrome', 'browserVersion': 'latest', 'platformName': 'WIN11', 'tb:options': { 'key': 'API_KEY', 'secret': 'API_SECRET' } }; (async function example() { const driver = await new Builder() .usingServer('https://hub.testingbot.com/wd/hub') .withCapabilities(capabilities) .setChromeOptions(chromeOptions) .build(); await driver.get('https://permission.site/#read-text'); await driver.findElement(By.id('read-text')).click(); await driver.sleep(2000); await driver.quit(); })(); ``` ```ruby require 'selenium-webdriver' options = Selenium::WebDriver::Chrome::Options.new options.add_preference("profile.content_settings.exceptions.clipboard", { '*' => { 'setting' => 1 } }) options.add_option("platformName", "WIN11") options.add_option("browserVersion", "latest") options.add_option("tb:options", { key: "API_KEY", secret: "API_SECRET" }) driver = Selenium::WebDriver.for :remote, url: "https://hub.testingbot.com/wd/hub", options: options driver.navigate.to "https://permission.site/#read-text" driver.find_element(:id, "read-text").click sleep(2) driver.quit ``` ```php setExperimentalOption('prefs', [ 'profile.content_settings.exceptions.clipboard' => [ '*' => ['setting' => 1] ] ]); $caps = DesiredCapabilities::chrome(); $caps->setCapability(ChromeOptions::CAPABILITY, $options); $caps->setCapability('platformName', 'WIN11'); $caps->setCapability('browserVersion', 'latest'); $caps->setCapability('tb:options', [ 'key' => 'API_KEY', 'secret' => 'API_SECRET' ]); $driver = RemoteWebDriver::create( "https://hub.testingbot.com/wd/hub", $caps, 120000 ); $driver->get("https://permission.site/#read-text"); $driver->findElement(WebDriverBy::id("read-text"))->click(); sleep(2); $driver->quit(); ?> ``` ```csharp using System; using System.Collections.Generic; using System.Threading; using OpenQA.Selenium; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Remote; namespace SampleCSharp { public class Program { static void Main(string[] args) { ChromeOptions options = new ChromeOptions(); options.PlatformName = "WIN11"; options.BrowserVersion = "latest"; var clipboardException = new Dictionary { ["[*.]permission.site,*"] = new Dictionary { ["last_modified"] = DateTimeOffset.Now.ToUnixTimeMilliseconds(), ["setting"] = 1 } }; options.AddUserProfilePreference("profile.content_settings.exceptions.clipboard", clipboardException); var tbOptions = new Dictionary { ["key"] = "API_KEY", ["secret"] = "API_SECRET" }; options.AddAdditionalOption("tb:options", tbOptions); IWebDriver driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), options); driver.Navigate().GoToUrl("https://permission.site/#read-text"); driver.FindElement(By.Id("read-text")).Click(); Thread.Sleep(2000); driver.Quit(); } } } ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/test-file-downloads # Test File Downloads TestingBot provides custom Selenium WebDriver commands to verify if a file was downloaded during a test automation session. Below you'll find examples on how to: - [Download files on remote desktop instances during your test](https://testingbot.com#download) - [Verify that the file was downloaded successfully](https://testingbot.com#verify) - [Retrieve (meta) properties from the downloaded file](https://testingbot.com#retrieve) - [Retrieve the contents of the downloaded file](https://testingbot.com#contents) All functions below can be used with a `fileName` argument, or without an argument. If you do not specify a specific filename, TestingBot will automatically select the most recent downloaded file. ## Download files on remote desktop instances during your test As part of your test automation flow, you might want to test download functionality of your webapp. In the example below, we'll first show you how to instruct the remote TestingBot browser to download a file on the TestingBot virtual machine. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java ChromeOptions options = new ChromeOptions(); options.setPlatformName("WIN11"); options.setBrowserVersion("latest"); HashMap tbOptions = new HashMap<>(); tbOptions.put("key", "API_KEY"); tbOptions.put("secret", "API_SECRET"); tbOptions.put("name", "My First Download Test"); options.setCapability("tb:options", tbOptions); WebDriver driver = new RemoteWebDriver(new URL("https://hub.testingbot.com/wd/hub"), options); driver.get("https://testingbot.com/security"); driver.findElement(By.cssSelector("body > div.bigHeader.enterprise > div.hero.enterprise.feature > div > div > div > a")).click(); Thread.sleep(1000); driver.quit(); ``` ```python import time from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.chrome.options import Options options = Options() options.browser_version = 'latest' options.platform_name = 'WIN11' options.set_capability('tb:options', { 'key': 'API_KEY', 'secret': 'API_SECRET', 'name': 'My First Download Test' }) driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options ) driver.get("https://testingbot.com/security") driver.find_element(By.CSS_SELECTOR, "body > div.bigHeader.enterprise > div.hero.enterprise.feature > div > div > div > a").click() time.sleep(2) driver.quit() ``` ```javascript const { Builder, By } = require('selenium-webdriver'); const chrome = require('selenium-webdriver/chrome'); const fs = require('fs'); async function main() { let options = new chrome.Options(); options.setBrowserVersion('latest'); options.setPlatformName('WIN11'); options.set('tb:options', { 'key': 'API_KEY', 'secret': 'API_SECRET', 'name': 'My First Download Test' }); let driver = await new Builder() .usingServer('https://hub.testingbot.com/wd/hub') .setChromeOptions(options) .build(); await driver.get('https://testingbot.com/security'); await driver.findElement(By.css("body > div.bigHeader.enterprise > div.hero.enterprise.feature > div > div > div > a")).click(); await driver.sleep(2000); await driver.quit(); } main(); ``` ```ruby #!/usr/bin/env ruby require 'rubygems' require 'selenium-webdriver' options = Selenium::WebDriver::Chrome::Options.new options.browser_version = 'latest' options.platform_name = 'WIN11' options.add_option('tb:options', { 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'My First Download Test' }) driver = Selenium::WebDriver.for( :remote, url: 'https://hub.testingbot.com/wd/hub', options: options ) driver.navigate.to 'https://testingbot.com/security' # click the download button element = driver.find_element(:css, 'body > div.bigHeader.enterprise > div.hero.enterprise.feature > div > div > div > a') element.click sleep 6 driver.quit ``` ```php require_once('vendor/autoload.php'); use Facebook\WebDriver\Remote\RemoteWebDriver; use Facebook\WebDriver\Remote\DesiredCapabilities; use Facebook\WebDriver\Chrome\ChromeOptions; use Facebook\WebDriver\WebDriverBy; $options = new ChromeOptions(); $capabilities = DesiredCapabilities::chrome(); $capabilities->setCapability(ChromeOptions::CAPABILITY, $options); $capabilities->setCapability('browserVersion', 'latest'); $capabilities->setCapability('platformName', 'WIN11'); $capabilities->setCapability('tb:options', [ 'key' => 'API_KEY', 'secret' => 'API_SECRET', 'name' => 'My First Download Test' ]); $web_driver = RemoteWebDriver::create( 'https://hub.testingbot.com/wd/hub', $capabilities ); $web_driver->get('https://testingbot.com/security'); $web_driver->findElement(WebDriverBy::cssSelector('body > div.bigHeader.enterprise > div.hero.enterprise.feature > div > div > div > a'))->click(); $web_driver->quit(); ``` ```csharp using System; using System.Collections.Generic; using OpenQA.Selenium; using OpenQA.Selenium.Remote; using OpenQA.Selenium.Chrome; using System.IO; using System.Threading; namespace SeleniumTest { class FileDownload { static void Main(string[] args) { ChromeOptions options = new ChromeOptions(); options.BrowserVersion = "latest"; options.PlatformName = "WIN11"; options.AddAdditionalOption("tb:options", new Dictionary { ["key"] = "API_KEY", ["secret"] = "API_SECRET", ["name"] = "My First Download Test" }); IWebDriver driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), options ); driver.Navigate().GoToUrl("https://testingbot.com/security"); driver.FindElement(By.CssSelector("body > div.bigHeader.enterprise > div.hero.enterprise.feature > div > div > div > a")).Click(); Thread.Sleep(2000); driver.Quit(); } } } ``` ## Verify that the file was downloaded successfully Once you've downloaded a file during your test automation, you might want to check if it was downloaded correctly (does it exist on the disk?). To perform this check, TestingBot has created a custom `tb:fileExists` command which can be used with the JavascriptExecutor provided by Selenium WebDriver. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java Map fileMap = new HashMap<>(); fileMap.put("fileName", "...the file name..."); ((JavascriptExecutor) driver).executeScript("tb:fileExists", fileMap); ``` ```python driver.execute_script("tb:fileExists", { "fileName": "...the file name..." }) ``` ```javascript await driver.executeScript('tb:fileExists', { "fileName": "...the file name..." }); ``` ```ruby driver.execute_script('tb:fileExists', { 'fileName' => '...the file name...' }) ``` ```php $driver->executeScript('tb:fileExists', [ 'fileName' => '...the file name...' ]); ``` ```csharp ((IJavaScriptExecutor)driver).ExecuteScript("tb:fileExists", new Dictionary { ["fileName"] = "...the file name..." }); ``` ## Retrieve (meta) properties from the downloaded file You can check the integrity of the downloaded file during your test, with the `tb:fileProperties` command. This will return the following file (meta) data: ```json { "changed_time": 1649947630, "created_time": 1649947629, "modified_time": 1649947630, "size": 114123, "md5": "5d4f7a7d4335c99048c3cc7563ffa7bf" } ``` - `changed_time`: Indicates, in unix epoch time, when the file (permission or content) was changed. - `created_time`: Indicates, in unix epoch time, when the file was downloaded. - `modified_time`: Indicates, in unix epoch time, when the file content was changed. - `size`: The size of the downloaded file, in bytes. - `md5`: The MD5 checksum of the downloaded file. Useful to verify the integrity of the file. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java Map fileMap = new HashMap<>(); fileMap.put("fileName", "...the file name..."); ((JavascriptExecutor) driver).executeScript("tb:fileProperties", fileMap); ``` ```python driver.execute_script("tb:fileProperties", { "fileName": "...the file name..." }) ``` ```javascript await driver.executeScript('tb:fileProperties', { "fileName": "...the file name..." }); ``` ```ruby driver.execute_script('tb:fileProperties', { 'fileName' => '...the file name...' }) ``` ```php $driver->executeScript('tb:fileProperties', [ 'fileName' => '...the file name...' ]); ``` ```csharp ((IJavaScriptExecutor)driver).ExecuteScript("tb:fileProperties", new Dictionary { ["fileName"] = "...the file name..." }); ``` ## Retrieve the contents of the downloaded file It is possible to download the file, which was downloaded on the remote TestingBot machine, to your own computer. TestingBot has created a custom command, `tb:fileContent`, which will return a Base64-encoded string of the file content. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java Map fileMap = new HashMap<>(); fileMap.put("fileName", "...the file name..."); String base64EncodedFile = (String) ((JavascriptExecutor) driver).executeScript("tb:fileContent", fileMap); byte[] data = Base64.getDecoder().decode(base64EncodedFile); OutputStream stream = new FileOutputStream("filename.txt"); stream.write(data); stream.close(); ``` ```python get_file_content = driver.execute_script('tb:fileContent', { 'fileName': '...the file name...' }) data = base64.b64decode(get_file_content) f = open('filename.txt', 'wb') f.write(data) ``` ```javascript const fs = require('fs'); let base64data = await driver.executeScript('tb:fileContent', { 'fileName': '...the file name...' }); let data = Buffer.from(base64data, 'base64'); fs.writeFileSync('filename.txt', data); console.log('File ready'); ``` ```ruby get_file_content = driver.execute_script('tb:fileContent', { 'fileName' => '...the file name...' }) decoded_content = Base64.decode64(get_file_content) f = File.open('filename.txt', 'wb') f.write(decoded_content) f.close ``` ```php $base64content = $driver->executeScript('tb:fileContent', [ 'fileName' => '...the file name...' ]); $decoded = base64_decode($base64content); file_put_contents('filename.txt', $decoded); ``` ```csharp string base64encode = (string) ((IJavaScriptExecutor)driver).ExecuteScript("tb:fileContent", new Dictionary { ["fileName"] = "...the file name..." }); byte[] b = Convert.FromBase64String(base64encode); File.WriteAllBytes(@"filename.txt", b); ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/auto-it # AutoIT AutoIT is a scripting language, with syntax similar to BASIC, used to automate programs on Windows. TestingBot has created a custom command, `tb:autoit`, which you can use during your Selenium WebDriver tests to automate parts of Windows and Windows GUI programs. AutoIT is compatible with all versions of Windows and provides many automation features, including: - Manipulate windows and processes - Interact with standard Windows components - Simulate keystrokes and mouse movements In the example below, we will show you a basic example of what an AutoIT script looks like. ```bash MsgBox(0, "My Title", "Hello from AutoIT") ``` ## Running AutoIT during a Selenium WebDriver test TestingBot has created a custom command, `tb:autoit`. You can use this command to send AutoIT scripts during your WebDriver test. You could for example use AutoIT scripts to automate a native dialog, for example a file upload dialog, opened by the browser during your WebDriver test. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java String autoItContent = "MsgBox(0, \"My Title\", \"Hello from AutoIT\")"; ((JavascriptExecutor) driver).executeScript("tb:autoit", autoItContent); ``` ```python autoItContent = "MsgBox(0, \"My Title\", \"Hello from AutoIT\")" driver.execute_script("tb:autoit", autoItContent) ``` ```javascript const autoItContent = 'MsgBox(0, "My Title", "Hello from AutoIT")'; await driver.executeScript("tb:autoit", autoItContent); ``` ```ruby payload = <<-EOF MsgBox(0, "My Title", "Hello from AutoIT") EOF driver.execute_script("tb:autoit", payload) ``` ```php $autoItScript = 'MsgBox(0, "My Title", "Hello from AutoIT")'; $driver->executeScript("tb:autoit", $autoItScript); ``` ```csharp ((IJavaScriptExecutor)driver).ExecuteScript("tb:autoit", "MsgBox(0, \"My Title\", \"Hello from AutoIT\")"); ``` ### Specifying a URL to AutoIT script Instead of specifying the AutoIT commands in the `tb:autoit` command, you can also specify a URL to a file that contains all the AutoIT commands: [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java ((JavascriptExecutor) driver).executeScript("tb:autoit", "https://testingbot.com/autoit/example.au3"); ``` ```python driver.execute_script("tb:autoit", "https://testingbot.com/autoit/example.au3") ``` ```javascript await driver.executeScript("tb:autoit", "https://testingbot.com/autoit/example.au3"); ``` ```ruby driver.execute_script("tb:autoit", "https://testingbot.com/autoit/example.au3") ``` ```php $driver->executeScript("tb:autoit", "https://testingbot.com/autoit/example.au3"); ``` ```csharp ((IJavaScriptExecutor)driver).ExecuteScript("tb:autoit", "https://testingbot.com/autoit/example.au3"); ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/osascript # AppleScript AppleScript (or osascript) is a scripting language which comes by default on macOS. It allows for automating various (native) macOS applications and can be used in combination with Selenium WebDriver via TestingBot's custom `tb:osascript` command. In the example below, you'll find a simple AppleScript example which will open Safari and enable automation. ```applescript tell application "Safari" launch activate tell application "System Events" set v to (value of attribute "AXMenuItemMarkChar" of menu item "Allow Remote Automation" of menu "Develop" of menu bar item "Develop" of menu bar 1 of application process "Safari") if v is missing value then click menu item "Allow Remote Automation" of menu "Develop" of menu bar item "Develop" of menu bar 1 of application process "Safari" else log "OK" end if end tell quit end tell ``` ## Running AppleScript during a Selenium WebDriver test TestingBot has created a custom command, `tb:osascript`, which you can use to pass in your specific AppleScript at any point during your (macOS) WebDriver test. For example, you can use AppleScript to automate a native dialog appearing during your test, such as a file upload dialog from the browser. You can specify multiple different AppleScripts at any point during your test. The session will wait until the script has ended, before proceeding to the other WebDriver commands. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java String appleScriptContent = "display alert \"Success!\""; Map osaMap = new HashMap<>(); osaMap.put("script", appleScriptContent); ((JavascriptExecutor) driver).executeScript("tb:osascript", osaMap); ``` ```python osa_script_content = "display alert \"Success!\"" driver.execute_script("tb:osascript", { "script": osa_script_content }) ``` ```javascript await driver.executeScript('tb:osascript', { "script": 'display alert "Success!"' }); ``` ```ruby applescript_content = <<-EOF display alert "Success!" EOF driver.execute_script("tb:osascript", { script: applescript_content }) ``` ```php $osaScript = 'display alert "Success!"'; $driver->executeScript("tb:osascript", ["script" => $osaScript]); ``` ```csharp var scriptArgs = new Dictionary { ["script"] = "display alert \"Success!\"" }; ((IJavaScriptExecutor)driver).ExecuteScript("tb:osascript", scriptArgs); ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/ie-mode-on-edge # IE mode on Edge browser Some (older) websites are built to run on Internet Explorer and might have features that are not supported by current browsers, such as Microsoft Edge. Microsoft Edge is the successor of Internet Explorer. Microsoft Edge provides an **Internet Explorer Mode** with Microsoft Edge, which allows you to run the Edge browser just like you would be running an IE 11 browser. IE mode on Microsoft Edge is supported from Selenium version 4 and higher. ## Start using Internet Explorer Mode To run your Selenium 4 tests with IE Mode and Edge, you can use the `ie.edgechromium` capability with `se:ieOptions`. Please see the example below. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java InternetExplorerOptions options = new InternetExplorerOptions(); options.attachToEdgeChrome(); HashMap tbOptions = new HashMap<>(); tbOptions.put("key", "API_KEY"); tbOptions.put("secret", "API_SECRET"); options.setCapability("tb:options", tbOptions); WebDriver driver = new RemoteWebDriver(new URL("https://hub.testingbot.com/wd/hub"), options); ``` ```python from selenium import webdriver from selenium.webdriver.ie.options import Options options = Options() options.attach_to_edge_chrome = True options.set_capability('tb:options', { 'key': 'API_KEY', 'secret': 'API_SECRET' }) driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options ) ``` ```javascript const { Builder } = require('selenium-webdriver'); const ie = require('selenium-webdriver/ie'); async function runTest() { let options = new ie.Options(); options.useEdgeChromium(true); options.set('tb:options', { 'key': 'API_KEY', 'secret': 'API_SECRET' }); let driver = await new Builder() .usingServer('https://hub.testingbot.com/wd/hub') .setIeOptions(options) .build(); } runTest(); ``` ```ruby options = Selenium::WebDriver::IE::Options.new options.attach_to_edge_chrome = true options.add_option('tb:options', { 'key' => 'API_KEY', 'secret' => 'API_SECRET' }) driver = Selenium::WebDriver.for(:remote, url: 'https://hub.testingbot.com/wd/hub', options: options ) ``` ```php $capabilities = DesiredCapabilities::internetExplorer(); $capabilities->setCapability('platformName', 'WIN11'); $capabilities->setCapability('se:ieOptions', [ 'ie.edgechromium' => true ]); $capabilities->setCapability('tb:options', [ 'key' => 'API_KEY', 'secret' => 'API_SECRET' ]); $driver = RemoteWebDriver::create( 'https://hub.testingbot.com/wd/hub', $capabilities ); ``` ```csharp var options = new InternetExplorerOptions(); options.AttachToEdgeChrome = true; options.PlatformName = "WIN11"; options.AddAdditionalOption("tb:options", new Dictionary { ["key"] = "API_KEY", ["secret"] = "API_SECRET" }); IWebDriver driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), options ); ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/selenium/grid # Selenium Grid and TestingBot Selenium Grid is an open-source project that provides distributed testing on a variety of browsers, across different machines called nodes. Setting up and maintaining a Selenium Grid takes time and effort. For every new browser version release, the configuration will need to be adjusted. You'll need to configure the appropriate driver (ChromeDriver, GeckoDriver, SafariDriver, EdgeDriver, ...), provision enough hardware and tune for performance. Running tests on TestingBot takes away the hassle of setting up and configuring a Selenium Grid yourself. TestingBot supports 100% compatibility with Selenium and Selenium Grid and has enhanced it with new useful features: - Screen recording and automated screenshots for every session. - Every test runs on a single-use VM, configured for performance and stability. - Integration with various [CI/CD solutions](https://testingbot.com/support/integrations/ci-cd). - Share test results and capacity with [other team members](https://testingbot.com/support/team/sub-accounts). - Insight into test failures with [Test Insights](https://testingbot.com/support/analytics/insights). ## Installing Selenium Grid To get started, please make sure you have a machine that has these things available: - Java JRE 11 - Download the [Selenium Server JAR file](https://github.com/SeleniumHQ/selenium/releases/latest) The machine will serve as a Selenium Grid endpoint. Tests will need to use this machine's IP address to forward test requests to. The Grid will then forward the requests to the appropriate nodes connected to the grid. To install the Grid, please follow the [Selenium Grid documentation](https://www.selenium.dev/documentation/grid/getting_started/). ## Running tests on Selenium Grid Once you have installed and configured your Selenium grid, you are ready to configure your tests to use the Selenium Grid. Below we'll show an example on how to change your existing tests from running on a local browser on your computer, to using the Selenium Grid you've just set up. [Local Execution](https://testingbot.com#) [Remote Execution](https://testingbot.com#) ```java WebDriver driver = new ChromeDriver(); driver.get("https://www.google.com/ncr"); WebElement element = driver.findElement(By.name("q")); element.sendKeys("TestingBot"); element.submit(); driver.quit(); ``` ```java WebDriver driver = new RemoteWebDriver(new URL("http://localhost:4444"), new ChromeOptions()); driver.get("https://www.google.com/ncr"); WebElement element = driver.findElement(By.name("q")); element.sendKeys("TestingBot"); element.submit(); driver.quit(); ``` In the example above we assume the Selenium Grid was set up on the same machine as the test you are running it from (`http://localhost:4444`). ## Extending Selenium Grid with TestingBot Once you have your own Selenium Grid running, you might want to extend it with browsers that are currently not available on your local network. Adding macOS, mobile browsers and physical devices takes a lot of effort. Expand your Selenium Grid with TestingBot's extensive grid of browsers and devices. Selenium Grid offers a **Relay** feature that enables a local Selenium Grid to add TestingBot as an extra node. This means your tests can target browsers and devices that are not available on your local Selenium Grid, while still using the nodes that are available from your local Grid. This results in a hybrid solution of using a local Grid, with low latency, and the TestingBot Grid with extensive coverage. ### Create a toml Configuration File Below is a sample `config.toml` configuration file: ```toml [node] detect-drivers = false [relay] url = "https://hub.testingbot.com/wd/hub" configs = [ "5", '{"browserName": "chrome", "platformName": "Windows 11", "browserVersion": "latest"}', "10", '{"browserName": "firefox", "platformName": "Windows 11", "browserVersion": "latest"}', "5", '{"browserName": "safari", "platformName": "macOS 15", "browserVersion": "18"}', "2", '{"browserName": "safari", "platformName": "iOS", "appium:platformVersion": "18.0", "appium:deviceName": "iPhone 16"}', "2", '{"browserName": "chrome", "platformName": "android", "appium:platformVersion": "15.0", "appium:deviceName": "Pixel 9"}' ] ``` This will add 5 extra browser combinations, through the use of the Selenium Grid `relay` function. The configuration determines the capabilities that will be relayed to TestingBot, together with the maximum number of concurrent tests per configuration. We can now start your local Selenium grid with the extra relay functionality: ```bash java -jar selenium-server-.jar node --config config.toml ``` Now your tests can use your own Selenium grid to run tests on your own nodes, as well as on TestingBot. ### Run a test on a Selenium Grid The following example shows how to run a test on your local Selenium Grid with forwarding to TestingBot. ```java SafariOptions browserOptions = new SafariOptions(); browserOptions.setPlatformName("macOS 15"); browserOptions.setBrowserVersion("18"); Map tbOptions = new HashMap<>(); tbOptions.put("key", System.getenv("TESTINGBOT_KEY")); tbOptions.put("secret", System.getenv("TESTINGBOT_SECRET")); browserOptions.setCapability("tb:options", tbOptions); RemoteWebDriver driver = new RemoteWebDriver(new URL("http://localhost:4444"), browserOptions); driver.get("https://www.google.com/ncr"); WebElement element = driver.findElement(By.name("q")); element.sendKeys("TestingBot"); element.submit(); driver.quit(); ``` Even though this example references your local Selenium grid, it will forward the test request to TestingBot if the capability is not available on your local grid. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/puppeteer # Puppeteer Testing [Puppeteer](https://pptr.dev/) is an automation framework which uses the Chrome Devtools protocol to automate a browser. With Puppeteer, you can control a headless (or headful) browser through the Devtools protocol. Compared to Selenium, this might be faster when running automated tests. Please see the documentation below on how to run Puppeteer tests on Chrome, Microsoft Edge and Firefox browsers in the TestingBot cloud. See our [Example Puppeteer Repository](https://github.com/testingbot/puppeteer-testingbot-example) for some examples on how to run Puppeteer tests on TestingBot. ## Installing Puppeteer To install Puppeteer, simply use `yarn` or `npm`: ```bash npm i --save puppeteer ``` By default, installing Puppeteer will also install a Chromium build. If you only want to use the TestingBot cloud, you might consider installing Puppeteer without the bundled Chromium build, called `puppeteer-core`: ```bash npm i puppeteer-core ``` ## Running your first Puppeteer test To run your first test, please use this example: [Chrome](https://testingbot.com#) [Edge](https://testingbot.com#) [Firefox](https://testingbot.com#) ```javascript const puppeteer = require('puppeteer-core') const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET }, browserName: 'chrome', browserVersion: 'latest' } ;(async () => { const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) const page = await browser.newPage() await page.goto('https://testingbot.com') await page.screenshot({ path: 'screenshot.png' }) await browser.close() })() ``` This example will start a Chrome Browser, navigate to TestingBot.com and save a PNG screenshot of the homepage. ```javascript const puppeteer = require('puppeteer-core') const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET }, browserName: 'edge', browserVersion: 'latest' } ;(async () => { const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) const page = await browser.newPage() await page.goto('https://testingbot.com') await page.screenshot({ path: 'screenshot.png' }) await browser.close() })() ``` This example will start a Microsoft Edge Browser, navigate to TestingBot.com and save a PNG screenshot of the homepage. ```javascript const puppeteer = require('puppeteer-core') const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET }, browserName: 'firefox', browserVersion: 'latest' } ;(async () => { const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) const page = await browser.newPage() await page.goto('https://testingbot.com') await page.screenshot({ path: 'screenshot.png' }) await browser.close() })() ``` This example will start a Firefox Browser, navigate to TestingBot.com and save a PNG screenshot of the homepage. ## Updating your existing Puppeteer scripts With a Puppeteer test, you'll usually start a browser with `await puppeteer.launch()` To start using our service, simply replace this line with our browser endpoint: ### Before ```javascript const browser = await puppeteer.launch() ``` ### After ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET }, browserName: 'chrome', browserVersion: 'latest' } const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` ## Specifying browser and version To specify on which browser and version your Puppeteer test should run, you can include both a `browserName` and `browserVersion` in the `browserWSEndpoint` URL. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Updating Test Details Every Puppeteer test will appear in the TestingBot dashboard, by default as an unnamed test. You can specify various parameters in the `browserWSEndpoint` to customize this. Specify a [name](https://testingbot.com/support/web-automate/puppeteer/options#name), [extra (metadata) information](https://testingbot.com/support/web-automate/puppeteer/options#customdata) or group tests together as a [build](https://testingbot.com/support/web-automate/puppeteer/options#build). TestingBot has created a custom Puppeteer command, which you can use in your tests, to mark a test as passed or failed in the TestingBot dashboard. See the example below, where we run a Puppeteer script and depending on whether the test passes or fails, we send back the correct test status to TestingBot. ```javascript const puppeteer = require('puppeteer-core') const expect = require('chai').expect ;(async () => { const browser = await puppeteer.connect({ browserWSEndpoint: 'wss://cloud.testingbot.com?key=API_KEY&secret=API_SECRET&browserName=chrome&browserVersion=latest&platform=WIN11' }) const page = await browser.newPage() await page.goto('https://testingbot.com/') const title = await page.title() try { expect(title).to.equal("Cross Browser Testing and Mobile App Testing. Test on web browsers, mobile emulators, simulators and physical mobile devices.", 'Expected page title is incorrect!') // This is ok, so mark the test as passed await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'setSessionStatus',arguments: {passed: true, reason: 'Title matched'}})}`) } catch { // Test failed, mark the test as failed on TestingBot await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'setSessionStatus',arguments: {passed: false, reason: 'Title did not match'}})}`) } await browser.close() })() ``` ## Puppeteer and TestingBot Tunnel Using Puppeteer in combination with [TestingBot Tunnel](https://testingbot.com/support/tunnel) allows you to test websites behind a firewall or on your local machine with Puppeteer on TestingBot. ### Setup Tunnel To use TestingBot Tunnel in combination with Puppeteer, you first need to start the TestingBot Tunnel and pass it a `tunnelIdentifier` name of your liking. ```bash java -jar testingbot-tunnel.jar key secret -i myPuppeteerTunnel ``` In this example, we'll start a TestingBot Tunnel with identifier **myPuppeteerTunnel**. Next, we'll need to configure our Puppeteer script to use the tunnel, using the same identifier. ### Setup Puppeteer To let Puppeteer know which tunnel we want to use, we need to specify the identifier from before as a parameter. Please see the example below where we pass the `tunnelIdentifier` with **myPuppeteerTunnel**. ```javascript const puppeteer = require('puppeteer') const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET }, browserName: 'chrome', browserVersion: 'latest', tunnelIdentifier: 'myPuppeteerTunnel' } ;(async () => { const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}`, headless: false }) const page = await browser.newPage() await page.goto('http://localhost:3000') await page.screenshot({ path: 'screenshot.png' }) await browser.close() })() ``` This Puppeteer script will open the latest Chrome browser on the TestingBot browser grid, use the tunnel to access your localhost website and take a screenshot from that page. ### Puppeteer and TestingBot Tunnel Launcher [testingbot-tunnel-launcher](https://github.com/testingbot/testingbot-tunnel-launcher) is a NodeJS library which allows you to manage the tunnel from within your NodeJS script. To use it, you can install with `npm install testingbot-tunnel-launcher` and use it with Puppeteer: ```javascript const testingbotTunnel = require('testingbot-tunnel-launcher') const puppeteer = require('puppeteer-core') ;(async () => { const tunnel = await new Promise((resolve, reject) => { testingbotTunnel({ apiKey: 'API_KEY', apiSecret: 'API_SECRET', verbose: true, tunnelIdentifier: 'puppeteerTunnel' }, function (err, tunnel) { if (err) { reject(err) return } resolve(tunnel) }) }) const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET }, browserName: 'chrome', browserVersion: 'latest', tunnelIdentifier: 'puppeteerTunnel' } const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) const page = await browser.newPage() await page.goto('http://localhost:8080') await page.screenshot({ path: 'screenshot.png' }) await browser.close() await new Promise((resolve, reject) => { tunnel.close((err) => { if (err) { reject(err) return } resolve() }) }) })() ``` This example will start a new Tunnel, start a Puppeteer script on the latest Chrome version in TestingBot, and take a screenshot of the website running on your local computer `(http://localhost:8080)` ## Debugging Tips Below are some tips on how to debug your Puppeteer scripts. ### Verbose Logging To see logs of what Puppeteer is sending and receiving, you can use this environment variable: `env DEBUG="puppeteer:*"` ```bash DEBUG="puppeteer:*" node puppeteer.js ``` ### Slow Motion option With the `slowMo` option you can slow down each operation during the Puppeteer session. To see the result, please specify these options: ```javascript const puppeteer = require('puppeteer') const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET }, browserName: 'chrome', browserVersion: 'latest' } ;(async () => { const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}`, headless: false, slowMo: 250 // slow down by 250ms }) const page = await browser.newPage() await page.goto('https://testingbot.com') await page.screenshot({ path: 'screenshot.png' }) await browser.close() })() ``` ## Puppeteer Framework examples - [CodeceptJS](https://testingbot.com/support/web-automate/puppeteer/codeceptjs) `CodeceptJS` is an end-to-end test framework capable of running Puppeteer tests. - [Jest](https://testingbot.com/support/web-automate/puppeteer/jest) `Jest-Puppeteer` allows you to run tests with Jest on browsers controlled with Puppeteer. - [Chromedp](https://testingbot.com/support/web-automate/puppeteer/chromedp) `chromedp` is a Golang framework to use with Puppeteer. - [PyTest](https://testingbot.com/support/web-automate/puppeteer/pytest) `pytest-pyppeteer` is a Python framework to use with Puppeteer. - [WebdriverIO](https://testingbot.com/support/web-automate/puppeteer/webdriverio) `WebdriverIO` is an Automation Test Framework for Node.js. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/puppeteer/live-view # Test Live View When running automated Puppeteer tests with TestingBot, you can see a realtime view of the test while it is running. If you are running the tests in a CI/CD pipeline, or on your own machines, it might be useful to see what is happening during the test execution. TestingBot offers a realtime live view of your Puppeteer tests, so you can see what is happening during the test execution. You can embed this live view in your own dashboards, or open it in a separate window. The live view also supports human in the loop: you can take control of the browser during the test execution, with mouse and keyboard. Realtime Views can be useful for: - Debugging tests by seeing what is happening during the test execution or share a view/control link with colleagues. - Human in the loop: instantly take control or provide input during a test. For example if the test gets stuck waiting for a Captcha or user input (for example authentication). - Embedding in other applications or dashboards for a unified view of test progress. ## Getting started You will need to generate a specific URL to access the live view of your test. To generate the URL, you will need the `sessionId` of the test that is running. To fetch the sessionId in your Puppeteer script, you can use the custom TestingBot command: ```javascript const testingBotResponse = await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'getSessionDetails'})}`) ``` The response will contain a `sessionId`, which is a unique identifier for this test. Once you have the sessionId, you can generate an authentication token as follows: The authentication hash is created with MD5. Please see the examples below. [NodeJS](https://testingbot.com#) [Java](https://testingbot.com#) [Python](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) ```ruby require 'testingbot' api = TestingBot::Api.new(API_KEY, API_SECRET) api.get_authentication_hash(sessionId) ``` ```java TestingbotREST restApi = new TestingbotREST("API_KEY", "API_SECRET"); String hash = restApi.getAuthenticationHash(String sessionId); ``` ```php $api = new TestingBot\TestingBotAPI($apiKey, $apiSecret); $api->getAuthenticationHash($sessionId); ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(API_KEY, API_SECRET) tb.get_share_link(sessionId) ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "API_KEY", api_secret: "API_SECRET" }); const hash = api.getAuthenticationHashForSharing(sessionId); ``` Now that you have the authentication hash, you can construct the live view URL as follows: ``` https://testingbot.com/tests//live?auth= ``` ## Embed Once you've constructed the live view URL, you can embed it in an iframe within your own application or dashboard: ```html ``` Adjust the width and height attributes as needed to fit your layout. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/puppeteer/capabilities # Puppeteer Capabilities Generator Generate Puppeteer connection options for your automated tests. Choose your browser, device emulation and configuration options to get the exact code you need for your Puppeteer tests. [![Selenium capabilities](https://testingbot.com/assets/support/selenium-266ecc0599068fc6eca0bcd9174fb46053c95cebe1c66f7a50c480f3f1a52929.svg) Selenium ](https://testingbot.com/support/web-automate/selenium/capabilities) [![Appium capabilities](https://testingbot.com/assets/support/appium-4b35a8bb56bae236bd84c539cee9b8bbb489a55ddf48e32d059c86685d230354.svg) Appium ](https://testingbot.com/support/app-automate/appium/capabilities) [![Puppeteer capabilities](https://testingbot.com/assets/support/puppeteer-0a8cdd38af69dabd46638afbb3bffc8a372902949aa599663ec379f3e875263c.svg) Puppeteer ](https://testingbot.com/support/web-automate/puppeteer/capabilities) [![Playwright capabilities](https://testingbot.com/assets/support/playwright-2dcb4e75b55ac0d1135cfa47be6137b13f46a2047ce78b96a34f51b0e2cc0d96.svg) Playwright ](https://testingbot.com/support/web-automate/playwright/capabilities) Browser Environment ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. Resolution 800x600 1024x768 1280x800 1280x960 1280x1024 1440x900 1680x1050 1600x1200 1920x1200 1920x1080 2560x1440 ## Project Capabilities Test Name Allows you to customize the name of the test, as it will appear in the dashboard. Build Name Group tests under the same build identifier for easy retrieval. ## Debugging Record Video Yes No ## Customization Headless Mode Run the browser without rendering UI for faster test execution. Yes No Geolocation Testing Run the test from a different geographical location. None Random Country Australia Belgium Brazil Canada Chile France Germany India Italy Japan Norway Singapore South Africa Sweden Switzerland United Arab Emirates United Kingdom United States Timezone ([list of timezones](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones)) Prerun Script ([documentation](https://testingbot.com/support/web-automate/selenium/test-options#prerun)) Upload URLs Looking for more options? See [all Puppeteer capabilities](https://testingbot.com/support/web-automate/puppeteer/options). ```javascript const puppeteer = require('puppeteer'); (async () => { const capabilities = { browserName: 'chrome', browserVersion: 'latest', platformName: 'Windows 11', 'tb:options': { 'key': process.env.TB_KEY, 'secret': process.env.TB_SECRET } } const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }); const page = await browser.newPage(); await page.goto('https://testingbot.com'); // Your test code here await browser.close(); })(); ``` --- URL: https://testingbot.com/support/web-automate/puppeteer/pytest # Puppeteer & PyTest `pytest-pyppeteer` is a Python framework to use with Puppeteer. Together with `pytest-asyncio`, you can write asynchronous PyTest tests which connect to the TestingBot browser grid. Run tests in parallel on the TestingBot browser grid with PyTest, Pyppeteer and pytest-asyncio. To get started, please install these packages: ```bash pip install pytest-asyncio pyppeteer ``` Now you're ready to use Python with Puppeteer. There's some more information about Pyppeteer on their [documentation pages](https://miyakogi.github.io/pyppeteer/). ## Example To get started, please see this simple example below. This will start a new Chrome browser in the TestingBot browser grid, and allow PyTest to control the browser via Puppeteer. ```python import pytest import pyppeteer @pytest.mark.asyncio async def test_title(): try: browser = await pyppeteer.connect(browserWSEndpoint='wss://cloud.testingbot.com?key=API_KEY&secret=API_SECRET&browserName=chrome&browserVersion=latest') page = await browser.newPage() await page.goto('https://testingbot.com') title = await page.title() assert title == 'Cross Browser Testing and Mobile App Testing | TestingBot' await browser.close() except Exception as E: print(E) ``` This example will open a Chrome browser, navigate to the TestingBot homepage and verify the page's title. We use `@pytest.mark.asyncio` to let PyTest know that we're using `await`. Please see [pytest-asyncio](https://github.com/pytest-dev/pytest-asyncio) for more information about event loops, awaits and fixtures. ## Specifying browser and version To specify on which browser and version your PyTest + Puppeteer test should run, you can include both a `browserName` and `browserVersion` in the `browserWSEndpoint` URL. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Parallel Testing with Pytest One of the great advantages of our service is that you can run multiple tests simultaneously (in parallel). This drastically shortens the total duration of your test suite, as multiple tests will run concurrently. We recommend using [pytest-parallel](https://pypi.org/project/pytest-parallel/). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/puppeteer/jest # Puppeteer & Jest Jest-Puppeteer allows you to run tests with Jest on browsers controlled with Puppeteer. To get started, please install these packages: ```bash npm install --save-dev jest-puppeteer puppeteer jest ``` Next, specify these settings in your Jest configuration file (`jest.config.js`): ```javascript module.exports = { rootDir: '.', testTimeout: 20000, testMatch: [ '/*.spec.js' ], preset: 'jest-puppeteer' } ``` ## Configure Jest with Puppeteer To configure Jest Puppeteer, create a new file called `jest-puppeteer.config.js` and add this to the file: ```javascript module.exports = { connect: { browserWSEndpoint: 'wss://cloud.testingbot.com?key=API_KEY&secret=API_SECRET&browserName=chrome&browserVersion=latest' } } ``` This instructs Jest and Puppeteer to connect to the TestingBot browser grid. ## Specifying browser and version To specify on which browser and version your Jest + Puppeteer test should run, you can include both a `browserName` and `browserVersion` in the `browserWSEndpoint` URL. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Run your first test To create a test, create a new file called `sample.spec.js` and add this to the file: ```javascript describe('Google', () => { beforeAll(async () => { await page.goto('https://google.com') }) it('should display google text on page', async () => { await expect(page).toMatch('google') }) }) ``` Now you can run your first test with Puppeteer on the TestingBot cloud: ```bash jest ``` This will open Google in a Chrome browser and verify if the word 'google' is on the page. You should see something like this in your terminal after running the command: ``` PASS ./sample.spec.js (6.503s) Google ✓ should display google text on page (81ms) Test Suites: 1 passed, 1 total Tests: 1 passed, 1 total ``` This test will also appear in your [Member Dashboard](https://testingbot.com/members). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/puppeteer/recorder # Introduction Since Chrome 89, Chrome DevTools has an option to record/generate Puppeteer scripts by recording the actions that you do in your Chrome browser. At the time of writing this documentation, this feature is still set as a Chrome DevTools experiment. To enable the experiment, check the `Recorder` checkbox under `Settings` \> `Experiments` in your Chrome Devtools. ## Example To start recording, go to the **Sources** panel in Chrome DevTools. Then select the **Recording** tab on the left pane: ![puppeteer recording](https://testingbot.com/assets/support/puppeteer/tab-3155a96aa174390eae3f10d0f48e0b3c27ea509afbdfecf105257c6ca276593d.webp) You can now add a new recording and give it a name: Finally, you're ready to start recording. Click on the **Record** button at the bottom to start recording your interactions. Click on the Record button again to stop the recording. Chrome DevTools will generate a Puppeteer script. You can easily modify this script to have it connect to the TestingBot browser grid: ![puppeteer recording](https://testingbot.com/assets/support/puppeteer/record-5ac6d036099e2cd5ec5533006cc3eb57739bfc625fa5082dc4b95a02bc8344f3.webp) Now you can make a simple change, to have your puppeteer script point to our browser grid: ### Before ```javascript const browser = await puppeteer.launch() ``` ### After ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET }, browserName: 'chrome', browserVersion: 'latest' } const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/puppeteer/chromedp # Chromedp chromedp is a golang package which allows you to automate a Chrome browser with Golang using Puppeteer syntax. To find out more about this package, please see the [chromedp documentation](https://pkg.go.dev/github.com/chromedp/chromedp). ## Installation Installing chromedp is very easy: ```bash go get -u github.com/chromedp/chromedp ``` ## Your first Test To get started, please see this simple example: ```golang package main import ( "context" "flag" "github.com/chromedp/chromedp" "log" ) func main() { var devToolWsUrl string var title string flag.StringVar(&devToolWsUrl, "devtools-ws-url", "wss://cloud.testingbot.com?key=API_KEY&secret=API_SECRET&browserName=chrome&browserVersion=latest", "DevTools Websocket URL") flag.Parse() actxt, cancelActxt := chromedp.NewRemoteAllocator(context.Background(), devToolWsUrl) defer cancelActxt() ctxt, cancelCtxt := chromedp.NewContext(actxt) // create new tab defer cancelCtxt() // close tab afterwards if err := chromedp.Run(ctxt, chromedp.Navigate("https://testingbot.com"), chromedp.Title(&title), ); err != nil { log.Fatalf("Failed getting title of testingbot.com: %v", err) } log.Println("Got title of:", title) } ``` This will connect with a Chrome browser running in the TestingBot cloud, go to the TestingBot homepage and print its title. ## Specifying browser and version To specify on which browser and version your Chromedp test should run, you can include both a `browserName` and `browserVersion` in the `browserWSEndpoint` URL. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/puppeteer/options # Puppeteer Options This page lists various options that you can use to customise your Puppeteer sessions on TestingBot. The [Puppeteer Capabilities generator](https://testingbot.com/support/web-automate/puppeteer/capabilities) allows you to easily generate the necessary capabilities for your tests. ## Basic Settings ### Browser Name The name of the browser to run your Puppeteer test on. TestingBot currently supports these browserNames: - **Chrome** - **Edge** - **Firefox** [Chrome](https://testingbot.com#) [Edge](https://testingbot.com#) [Firefox](https://testingbot.com#) ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET }, browserName: 'chrome', browserVersion: 'latest' } const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET }, browserName: 'edge', browserVersion: 'latest' } const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET }, browserName: 'firefox', browserVersion: 'latest' } const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` ### Browser Version The version of the browser to run your Puppeteer test on. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. Other Options: - `"version": "*"` : If you use `*` as version, TestingBot will pick a random version. - `"version": "latest"` : TestingBot will automatically take the latest version. You can also use `latest-1`, `latest-2`, ... to test on the next most recent versions. For example, if the current latest Chrome version is 89 and you use `latest-2`, then the test will run on Chrome 87. - `"version": "<=16"` : TestingBot will pick a version smaller than or equal to the version you specify with `<=`. `"version": ">=16"` : TestingBot will pick a version higher than or equal to the version you specify with `>=`. ### Browser Platform The OS/platform for the test VM which will run the Puppeteer test. ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET }, platform: 'WIN11' } const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` Possible values: - `WIN11` - `WIN10` - `TAHOE` - `SEQUOIA` - `SONOMA` - `VENTURA` - `MONTEREY` - `BIGSUR` - `LINUX` ### Make a video of your tests By default we record a video of your test, which is accessible in the member area. If you do not wish to have this, you can disable it with this option. Video should not slow down your test considerably. ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, screenrecorder: false }, browserName: 'chrome', browserVersion: 'latest' } const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` Value Type | Default Value || boolean | true | ### Test Privacy Make the test results for this test public so that everyone can access the results. ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, public: true }, browserName: 'chrome', browserVersion: 'latest' } const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` Value Type | Default Value || boolean | false | ### Blacklist hostnames The hostnames you specify will be pointed to localhost instead of their real destination. This means you can speed up tests by blocking third party content which you don't need and slows down your test. ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, blacklist: 'site1.com,site2.com' }, browserName: 'chrome', browserVersion: 'latest' } const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` Value Type || string (comma-separated) | ### Customize Logging By default, TestingBot records logs of all Puppeteer actions. Set this option to `false` if you don't want TestingBot to record anything (for example, if you have sensitive data). ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, recordLogs: true }, browserName: 'chrome', browserVersion: 'latest' } const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` Value Type | Default Value | Possible Values: || boolean | true | `true`, `false` | ### Custom Time Zones Change the Time Zone of the Virtual Machine to the Time Zone you specify. You can find a [list of timezones on Wikipedia](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones). Only location names are supported (not their paths). See some examples below: ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, timeZone: 'Etc/UTC' }, browserName: 'chrome', browserVersion: 'latest' } const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` Value Type | Default Value | Possible Values: || string | "Etc/UTC" | [List of timezones](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) | ### Change Screen Resolution Will adjust the screen resolution before your test. ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, 'screen-resolution': '1280x1024' }, browserName: 'chrome', browserVersion: 'latest' } const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` Value Type | Default Value | || string | "1280x1024" | Platform \| Resolutions \|\| Windows/Linux \| - 800x600 - 1024x768 - 1152x864 - 1280x768 - 1280x800 - 1280x960 - 1280x1024 - 1400x1050 - 1600x1200 - 1680x1050 - 1920x1080 - 1920x1200 - 2560x1440 \| \| macOS \| - 800x600 - 1024x768 - 1280x768 - 1280x800 - 1280x960 - 1280x1024 - 1366x768 - 1440x900 - 1600x900 - 1600x1200 - 1680x1050 - 1920x1080 - 1920x1200 - 2048x1536 \| | ### Geolocation Testing We provide an option where you can specify from which country you'd like to run the test from. Once you specify this option, the virtual machine we provision for your test will be configured to use a proxy in the country you specified. ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, geoCountryCode: 'US' }, browserName: 'chrome', browserVersion: 'latest' } const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` We currently support the following country codes: - **'\*'** : this will take a random country from the list below - **'AU'** : Australia - **'BE'** : Belgium - **'BR'** : Brazil - **'CA'** : Canada - **'CL'** : Chile - **'FR'** : France - **'DE'** : Germany - **'IN'** : India - **'IT'** : Italy - **'JP'** : Japan - **'NO'** : Norway - **'SG'** : Singapore - **'ZA'** : South Africa - **'SE'** : Sweden - **'CH'** : Switzerland - **'AE'** : United Arab Emirates - **'GB'** : United Kingdom - **'US'** : United States ### Upload This option allows you to upload a file to the test VM running the Puppeteer test. TestingBot will first download the file from the URL you specified and save it on the test VM so your test can use the file. ```javascript const puppeteer = require('puppeteer') const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, upload: 'https://testingbot.com/assets/logo-head.png', uploadFilepath: '/tmp/test.png' }, browserName: 'chrome', browserVersion: 'latest' } ;(async () => { const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) const page = await browser.newPage() await page.goto('http://localhost:3000') const elementHandle = await page.$('input[type=file]') await elementHandle.uploadFile('/tmp/test.png') await page.click('upload-button') await browser.close() })() ``` You need specify the download URL as `upload` and use `uploadFilepath` to indicate where TestingBot needs to put the file. ### Change Test Name Add a name to this test, which will show up in our member area and API. ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, name: 'First Test' }, browserName: 'chrome', browserVersion: 'latest' } const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` Value Type | Default Value || string | unnamed test | ### Group Tests A key you can use to group certain tests in the same build (for example in Jenkins). The builds will appear in our member area. ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, build: 'First Build' }, browserName: 'chrome', browserVersion: 'latest' } const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` Value Type || string | ### Timeout Controls the timeout setting of a Puppeteer session. This setting, which is specified in seconds, controls when a session times out. A timeout occurs when a session has not received any commands from your Puppeteer test for the amount of seconds specified (default is 90 seconds). If a certain Puppeteer command takes longer than the specified setting, the session will automatically close. ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, timeout: 90 }, browserName: 'chrome', browserVersion: 'latest' } const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` Value Type | Default Value || int (specify number of seconds) | 90 seconds | ### Idle Timeout The maximum amount of time a browser will wait before proceeding to the next step in your test. A timeout occurs when a session has not received any commands from your Puppeteer test for the xx amount of seconds specified. ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, timeout: 120000 }, browserName: 'chrome', browserVersion: 'latest' } const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` Value Type | Default Value || int (specify number of seconds) | 90 seconds | ### Maximum Test Duration The maximum duration for a single test. This is a safeguard to prevent bad tests from using up your credits. We generally recommend to keep tests short (less than 10 minutes). It's better to split up large tests in smaller individual tests. This keeps your tests fast and allows for more parallelization of your tests. ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, maxDuration: 1800 }, browserName: 'chrome', browserVersion: 'latest' } const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` Value Type | Default Value || int (specify number of seconds) | 1800 seconds (30 minutes) | ### Custom Metadata Send along custom data, for example your release, server, commit hash, ... This will show up on the test detail page in the TestingBot member area. ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, extra: 'Extra Information' }, browserName: 'chrome', browserVersion: 'latest' } const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` Value Type || string | ### Startup Flags You can customize the startup flags of the browser by passing in extra flags. This is identical to passing `args` with Puppeteer. ```javascript const puppeteer = require('puppeteer') const startupFlags = [] startupFlags.push('--window-size=1280,720') startupFlags.push('--hide-scrollbars') ;(async () => { const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com` + `?key=API_KEY&secret=API_SECRET&browserName=chrome&startupFlags=${JSON.stringify(startupFlags)}` }) const page = await browser.newPage() await page.goto('https://testingbot.com') await browser.close() })() ``` ### TestingBot Tunnel You can specify the `tunnelIdentifier` to [use Puppeteer with TestingBot Tunnel](https://testingbot.com/support/web-automate/puppeteer#tunnel). ```javascript const puppeteer = require('puppeteer') const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, tunnelIdentifier: 'myPuppeteerTunnel' }, browserName: 'chrome', browserVersion: 'latest' } ;(async () => { const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) const page = await browser.newPage() await page.goto('http://localhost:3000') await page.screenshot({ path: 'screenshot.png' }) await browser.close() })() ``` By default we support proxying these ports for localhost testing: `[443, 80, 8080, 3030, 3000, 3001, 3400]`. If you need additional ports, please pass an array of port numbers to either `localHttpPorts` or `localHttpsPorts` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/puppeteer/webdriverio # Puppeteer & WebdriverIO WebdriverIO is a popular Automation Test Framework for Node.js. It has built-in support for both `WebDriver` and the `Chrome DevTools Protocol`. To get started, please install WebdriverIO: ```bash npm i @wdio/cli @wdio/local-runner ``` Next, create a simple test suite with the command below. Follow the steps to set up a test suite. ```bash npx wdio config ``` This will generate some files that we can use to run a Puppeteer test with WebdriverIO on the TestingBot browser grid. ## Run your first test Create a new spec file with the example code below: ```javascript import { format } from 'util' import { remote } from 'webdriverio' (async () => { const browser = await remote({ capabilities: { 'wdio:devtoolsOptions': { browserWSEndpoint: format( `wss://cloud.testingbot.com?key=%s&secret=%s&browserName=chrome&browserVersion=latest`, process.env.TESTINGBOT_KEY, process.env.TESTINGBOT_SECRET ) } } }) await browser.url('https://testingbot.com') const title = await browser.getTitle() console.log(title) // returns "Cross Browser Testing and Mobile App Testing | TestingBot" await browser.deleteSession() })() ``` You can find more information about WebdriverIO's Puppeteer support in the [WebdriverIO documentation](https://webdriver.io/docs/automationProtocols#devtools-protocol). ## Specifying browser and version To specify on which browser and version your WebdriverIO + Puppeteer test should run, you can include both a `browserName` and `browserVersion` in the `browserWSEndpoint` URL. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/puppeteer/scraping # Puppeteer Scraping Tutorial Puppeteer is a framework that allows you to control a headless browser through scripting. The framework allows you to control a real browser, just like a normal user would. This means it's useful for both automated testing, as well as scraping. Scraping is an automated way to extract data from a website. TestingBot provides a [Scrape Function](https://testingbot.com/support/functions/scrape) to scrape a page without writing any code. ## Example To get started, please see the example below where we will scrape some text from the TestingBot website. This will start a new Chrome browser in the TestingBot browser grid and instruct the browser to navigate to the TestingBot website and scrape the text from a specific DOM element. ```javascript const puppeteer = require('puppeteer') const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET }, browserName: 'chrome', browserVersion: 'latest' } ;(async () => { const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) const page = await browser.newPage() await page.goto('https://testingbot.com') const title = await page.evaluate(() => { return document.querySelector('body > div.main > div.hero.home > div > div > p').textContent.trim() }) console.log(title); await browser.close() })() ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/puppeteer/codeceptjs # CodeceptJS & Puppeteer CodeceptJS is an end-to-end testing framework which provides support for many automation drivers, including Puppeteer. To get started, please install these packages: ```bash npm install codeceptjs puppeteer --save ``` To create a sample project from scratch, you can use this command: ```bash npx codeceptjs init ``` ## Configure CodeceptJS with Puppeteer To configure CodeceptJS, please edit `codecept.conf.js` and make sure to add the `Puppeteer` helper. ```javascript { // .. helpers: { Puppeteer: { url: "http://localhost", show: true, windowSize: '1200x900', chrome: { browserWSEndpoint: "wss://cloud.testingbot.com?key=API_KEY&secret=API_SECRET" } } } // .. } ``` The `browserWSEndpoint` URL instructs CodeceptJS to use TestingBot's Puppeteer grid to run the tests on. ## Specifying browser and version To specify on which browser and version your CodeceptJS + Puppeteer test should run, you can include both a `browserName` and `browserVersion` in the `browserWSEndpoint` URL. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Run your first test Let's create our first test, which will open the TestingBot website and check to see if the word 'TestingBot' is included on the page. ```javascript Feature('Test'); Scenario('test testingbot homepage', ({ I }) => { I.amOnPage('https://testingbot.com'); I.see('TestingBot'); }); ``` Now you can run this first test on TestingBot. ```bash npx codeceptjs run --steps ``` ``` Test -- test something I am on page "https://testingbot.com" I see "TestingBot" ✔ OK in 1057ms OK | 1 passed // 11s ``` This test will appear in your [Member Dashboard](https://testingbot.com/members), together with a video of the test and any logs that were generated. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/puppeteer/puppeteer-parallel-testing # Parallel Testing Parallel Testing means you can run multiple tests simultaneously, with Puppeteer. It allows you to speed up your test execution, drastically shortening your total test duration time. The documentation below will show you how to run Puppeteer tests concurrently on the TestingBot browser grid, which offers over 100+ Desktop Browser combinations for Puppeteer. ## Parallel Example In the example below, we'll start various Puppeteer sessions on different browser versions, in the TestingBot cloud. ```javascript const puppeteer = require('puppeteer-core') const doConnect = async (cap) => { const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET }, browserName: cap.browserName, browserVersion: cap.browserVersion, platform: cap.platform } const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) const page = await browser.newPage() await page.goto('https://testingbot.com') await page.screenshot({ path: 'screenshot.png' }) await browser.close() } const capabilities = [ { 'browserName': 'chrome', 'browserVersion': 'latest', 'platform': 'WIN11', }, { 'browserName': 'chrome', 'browserVersion': 'latest-1', 'platform': 'LINUX', }, { 'browserName': 'edge', 'browserVersion': 'latest', 'platform': 'SEQUOIA', }] ;(async () => { await Promise.all( capabilities.map((cap) => doConnect(cap)) ) console.log('All tests finished!'); })(); ``` You can use `latest` and `latest-x` to automatically run on the latest browser version. ### Queueing Depending on your account subscription, you can run multiple tests in parallel. If you exceed your maximum allocated slots, the connection will be queued until a slot frees up. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/puppeteer/browsers # Browser and OS versions available for Puppeteer Testing The TestingBot browser grid consists of +100 browser and OS combinations compatible with Puppeteer testing. We currently support these browser vendors with Puppeteer testing: - Chrome - Microsoft Edge - Mozilla Firefox ## Specifying browser combination To indicate on which browser you want your Puppeteer test to run, please use the following parameters as a `browserWSEndpoint` value. - `browserName` - `browserVersion` - `platform` Capability | Description | Possible Values: || `browserName` | The name of the browser | `chrome`, `edge` and `firefox` | | `browserVersion` | The version of the browser | - **Chrome:** 72 and above - **Edge:** 80 and above - **Firefox:** 110 and above You can use `latest`, `latest-1`, `latest-{x}` to test on the latest versions of the browser. | | `platform` | The Operating System where the test will run | - **Windows:** `WIN10`, `WIN11` - **macOS:** `TAHOE`, `SEQUOIA`, `SONOMA`, `VENTURA`, `MONTEREY`, `BIGSUR` | There was no release for version `82` of Chrome and Edge, so please do not specify this version. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/puppeteer/change-screen-resolution # Change desktop screen resolution with Puppeteer The TestingBot Desktop VMs (Windows, macOS and Linux) all support changing the screen-resolution before a test starts. We currently support the following screen resolutions: Value Type | Default Value | || string | "1280x1024" | Platform \| Resolutions \|\| Windows/Linux \| - 800x600 - 1024x768 - 1152x864 - 1280x768 - 1280x800 - 1280x960 - 1280x1024 - 1400x1050 - 1600x1200 - 1680x1050 - 1920x1080 - 1920x1200 - 2560x1440 \| \| macOS \| - 800x600 - 1024x768 - 1280x768 - 1280x800 - 1280x960 - 1280x1024 - 1366x768 - 1440x900 - 1600x900 - 1600x1200 - 1680x1050 - 1920x1080 - 1920x1200 - 2048x1536 \| | To change the screen resolution, please specify the `screen-resolution` parameter in the `browserWSEndpoint` URL. ```javascript const puppeteer = require('puppeteer-core') const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET }, browserName: 'chrome', browserVersion: 'latest', 'screen-resolution': '1600x900' } ;(async () => { const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) const page = await browser.newPage() await page.goto('https://testingbot.com') await page.screenshot({ path: 'screenshot.png' }) await browser.close() })() ``` Changing the screen resolution does not resize the viewport. To change the viewport with Puppeteer, please use `page.setViewport`. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/puppeteer/test-results # Puppeteer Test Report After running a Puppeteer test on TestingBot, you can see the test result in the [TestingBot Dashboard](https://testingbot.com#dashboard) overview. You can click each test to see a [detailed overview](https://testingbot.com#single) for each specific test. By default, a Puppeteer test runs in headless mode, so no video will be available. You can change this by [enabling video recording](https://testingbot.com/support/web-automate/puppeteer/options#screenrecorder). ## The Puppeteer Dashboard The dashboard contains an overview of all your previous test runs, together with some meta-data: - **Test Name** : you can pass this name via our [REST API](https://testingbot.com/support/web-automate/puppeteer#update). - **Pass/Failure state** : you can [indicate](https://testingbot.com/support/web-automate/puppeteer#update) whether this test passed or failed. - **Browser/Device** : the browser or device that was used by the test. TestingBot provides [more than 100 different browser/OS combinations](https://testingbot.com/support/web-automate/puppeteer/browsers). - **Start Date** : when we received the call to start your test. - **Test Duration** : how long it took for your test to run. This is the actual run time, potential queueing time is not included. ## Single Puppeteer Test Detail Page For each test, we have a test detail page, which shows an overview of all the data we have: - Test Name - [session details](https://testingbot.com/support/web-automate/puppeteer/get-session-details) of the test - the pass/failure state, if you used our [REST API](https://testingbot.com/support/web-automate/puppeteer#update) - the browser/device that was used by the test - the date when the test started - how long the test took to run - logs for the test - a video recording of the test (if the test ran with [recording enabled](https://testingbot.com/support/web-automate/puppeteer/options#screenrecorder)) - the user who started the test in your team ## Viewing results without the dashboard You can [share, embed and link to builds or individual tests](https://testingbot.com/support/other/sharing) without having to visit the dashboard. This is useful when you want to create your own reporting. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/puppeteer/debug-tests # Debug Failing Puppeteer Tests This document will guide you in using various techniques to debug Puppeteer tests running on TestingBot. We provide several features which can help you in locating problems or debugging Puppeteer tests. ## Interactive Session Our interactive session feature allows you to pause a running Puppeteer script, and inspect the remote browser with the included Developer Tools. Simply click the **Pause Test** button on the test page while the test is running and you will be navigated to a remote browser. You can now use your mouse and keyboard to interact with the browser and debug potential issues. One important thing to remember is that you will need to add a sleep/wait in your code, so that your Puppeteer commands stop being sent when debugging. ## Video Recording Every Puppeteer test that runs on TestingBot comes with a [recorded video](https://testingbot.com/support/web-automate/puppeteer/options#screenrecorder) of the entire screen, providing you run the Puppeteer test in `headful` mode. The video is available in our member area, where you can click each Puppeteer test. Or you can [fetch the video](https://testingbot.com/support/web-automate/puppeteer/get-session-details), together with other details, through our REST API. ## Log Files For every Puppeteer test running on TestingBot, we collect the browser's log. This log can be useful for pinpointing certain issues with your script. Log files are available in our member area, where you can click each Puppeteer test. Or you can [fetch the logs](https://testingbot.com/support/web-automate/puppeteer/get-session-details), together with other details, through our REST API. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/puppeteer/get-session-details # Get Puppeteer Session Details When a Puppeteer test runs on TestingBot, it will generate various artifacts such as logs, video, screenshots and other files. TestingBot will save these files for later retrieval, through our member dashboard or the TestingBot [REST API](https://testingbot.com/support/api). To fetch these details through our REST API, you will need to know the unique identifier for each test. TestingBot provides a `sessionId` which you can retrieve during your test, to use in REST API calls. ## Fetch the sessionId To fetch the sessionId in your Puppeteer script, you can use the custom TestingBot command: ```javascript const testingBotResponse = await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'getSessionDetails'})}`) ``` The response will contain a `sessionId`, which is a unique identifier for this test. You can then use this sessionId with the [TestingBot REST-API](https://testingbot.com/support/api). ```javascript const TestingBot = require('testingbot-api'); const tb = new TestingBot({ api_key: 'API_KEY', api_secret: 'API_SECRET' }) const puppeteer = require('puppeteer-core') ;(async () => { const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET }, browserName: 'chrome', browserVersion: 'latest' } const browser = await puppeteer.connect({ browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) const page = await browser.newPage() await page.goto('https://testingbot.com') const testingBotResponse = await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'getSessionDetails'})}`) const sessionId = testingBotResponse.sessionId const testData = { "test[success]" : "1", "test[status_message]" : "test" }; await new Promise((resolve) => tb.updateTest(testData, sessionId, function(error, testDetails) { resolve(testDetails) })) console.log(`Your unique TestingBot SessionId is ${sessionId}`) await browser.close() })() ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/puppeteer/mark-test-status # Mark test status with Puppeteer The Puppeteer tests you run on TestingBot are using the `CDP` protocol to send commands to a remote browser. These commands instruct the remote browser what to do: click, type, resize, ... Because the logic of your test cases is inside your test script, TestingBot has no way of knowing if a test failed or passed. There is a way to send back the test status to TestingBot, by including a code snippet at the end of your test, where you know whether the test passed or failed. Below we'll go over two methods to send back the test status to TestingBot, so that we can show the pass/failure state in the dashboard and in the [TestingBot analytics](https://testingbot.com/support/analytics/guide). ## Mark test status from within a test script TestingBot has created a custom Puppeteer command which you can use in your Puppeteer scripts. You can pass in a `boolean` to indicate whether the test **passed** or **failed**. ```javascript await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'setSessionStatus',arguments: { passed: false, reason: 'Title did not match' }})}`) ``` This command will set the test as failed, with a specific reason which will be visible on the TestingBot test detail page. You can pass in the reason for a failure with the `reason` field. This command will return a JSON object with the `sessionId` of the test, which can be used with our [REST-API](https://testingbot.com/support/api). You would typically call this at the end of each test, where you know if the test failed or passed. For example, see the code snippet below where we use `chai's expect` to verify if the test passed or not. ```javascript const puppeteer = require('puppeteer-core') const expect = require('chai').expect ;(async () => { const browser = await puppeteer.connect({ browserWSEndpoint: 'wss://cloud.testingbot.com?key=API_KEY&secret=API_SECRET&browserName=chrome&browserVersion=latest&platform=WIN11' }) const page = await browser.newPage() await page.goto('https://testingbot.com/') const title = await page.title() try { expect(title).to.equal("Cross Browser Testing and Mobile App Testing. Test on web browsers, mobile emulators, simulators and physical mobile devices.", 'Expected page title is incorrect!') // This is ok, so mark the test as passed await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'setSessionStatus',arguments: {passed: true, reason: 'Title matched'}})}`) } catch { // Test failed, mark the test as failed on TestingBot await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'setSessionStatus',arguments: {passed: false, reason: 'Title did not match'}})}`) } await browser.close() })() ``` ## Mark test status with API You can fetch the `sessionId` during an active Puppeteer test. This unique id can be used during or after the test to set the pass/fail state for the test. Marking the test as passed or failed will show the success state in the TestingBot test dashboard, on the individual test detail page and in the [TestingBot analytics](https://testingbot.com/support/analytics/guide) pages. To fetch the `sessionId`, please use the sample code below. ```javascript const testingBotResponse = await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'getSessionDetails'})}`) ``` The `testingBotResponse` variable will contain a `sessionId` property which you can use: ```json { "sessionId": "......" } ``` With the `sessionId`, you can now mark the test as passed or failed via the [TestingBot REST-API](https://testingbot.com/support/api): ```bash curl "https://api.testingbot.com/v1/tests/:sessionId" \ -X PUT \ -d "test[success]=1" \ -u API_KEY:API_SECRET ``` Learn more about this API call and other API calls on the TestingBot [API documentation](https://testingbot.com/support/api). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/puppeteer/browser-extension-testing # Testing Browser Extensions with Manifest V3 Testing browser extensions on Chrome with Puppeteer is possible with TestingBot. Please see the information below on how to load the extension before your test starts and how to perform UI testing with Puppeteer against your Manifest V3 extension(s). ## TestingBot Storage With [TestingBot Storage](https://testingbot.com/support/api/storage#upload), you can upload your extensions to the TestingBot servers. The advantage of this is that our test VMs can immediately download your extension from the TestingBot network, which is much faster than downloading from the public internet. The API call will return a unique identifier, similar to `tb://efe932jfk`. You can then use this identifier with the capability: `"load-extension" : "tb://..."` ## Chrome Browser Extension Testing with Puppeteer You can test your `.crx` extension by [uploading the extension](https://testingbot.com#storage) and using the URL of the extension with a `load-extension` capability. ```javascript const puppeteer = require('puppeteer-core') ;(async () => { const tbCapabilities = { browserName: 'chrome', platform: 'WIN11', key: process.env.TB_KEY, secret: process.env.TB_SECRET, 'load-extension': 'tb://path-to-extension' } const wsEndpoint = `wss://cloud.testingbot.com?capabilities=${encodeURIComponent( JSON.stringify(tbCapabilities) )}`; const browser = await puppeteer.connect({ browserWSEndpoint: wsEndpoint }) const page = await browser.newPage() await page.goto('https://testingbot.com') // add various checks to test your extension await browser.close() })(); ``` ### UI Testing Chrome Extensions You can test the UI and functionality of your extension in a remote Chrome browser. ```javascript const puppeteer = require('puppeteer-core') ;(async () => { const tbCapabilities = { browserName: 'chrome', platform: 'WIN11', key: process.env.TB_KEY, secret: process.env.TB_SECRET, 'load-extension': 'tb://path-to-extension' } const wsEndpoint = `wss://cloud.testingbot.com?capabilities=${encodeURIComponent( JSON.stringify(tbCapabilities) )}`; const browser = await puppeteer.connect({ browserWSEndpoint: wsEndpoint }) const page = await browser.newPage() await page.goto('chrome-extension://lliapakmekmldfaphofjcnbccemlgeoa/src/pages/popup/index') await browser.close() })(); ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/playwright # Playwright Testing [Playwright](https://playwright.dev/) is an open-source NodeJS library developed by Microsoft for automating browsers using the DevTools protocol. Its syntax is very similar to Puppeteer. The advantages of using Microsoft Playwright is that it offers various useful features: - **auto wait** : Playwright automatically waits for elements (locators) to appear in the DOM, reducing the need for manual waiting logic. - **browser compatibility** : Playwright supports multiple browsers: Chromium (Google Chrome, Microsoft Edge, and other Chromium-based browsers), Firefox and WebKit - **fast test execution** : tests (browser automation) will run much faster than Selenium, due to improved communication standards (WebSocket). ## Installing Playwright To install Playwright, simply use `yarn` or `npm`: ```bash npm i --save playwright ``` By default, installing Playwright will also install one or more browsers. If you only want to use the TestingBot cloud, you might consider installing Playwright without the bundled browsers, called `playwright-core`: ```bash npm i playwright-core ``` ## Running your first Playwright test To run your first test, please use this example: [Chrome](https://testingbot.com#) [Edge](https://testingbot.com#) [Firefox](https://testingbot.com#) [WebKit](https://testingbot.com#) ```javascript const playwright = require('playwright-core'); (async () => { const browser = await playwright.chromium.connect({ wsEndpoint: 'wss://cloud.testingbot.com/playwright?key=api_key&secret=api_secret&browserName=chrome&browserVersion=latest', }); const context = await browser.newContext(); const page = await context.newPage(); await page.goto('https://testingbot.com/'); await page.screenshot({ path: 'screenshot.png' }); await browser.close(); })(); ``` This example will start a Chrome Browser, navigate to testingbot.com and save a PNG screenshot of the webpage. ```javascript const playwright = require('playwright-core'); (async () => { const browser = await playwright.chromium.connect({ wsEndpoint: 'wss://cloud.testingbot.com/playwright?key=api_key&secret=api_secret&browserName=edge&browserVersion=latest', }); const context = await browser.newContext(); const page = await context.newPage(); await page.goto('https://testingbot.com/'); await page.screenshot({ path: 'screenshot.png' }); await browser.close(); })(); ``` This example will start a Microsoft Edge Browser, navigate to testingbot.com and save a PNG screenshot of the webpage. ```javascript const playwright = require('playwright-core'); (async () => { const browser = await playwright.firefox.connect({ wsEndpoint: 'wss://cloud.testingbot.com/playwright?key=api_key&secret=api_secret&browserName=firefox&browserVersion=latest', }); const context = await browser.newContext(); const page = await context.newPage(); await page.goto('https://testingbot.com/'); await page.screenshot({ path: 'screenshot.png' }); await browser.close(); })(); ``` This example will start a Mozilla Firefox Browser, navigate to testingbot.com and save a PNG screenshot of the webpage. ```javascript const playwright = require('playwright-core'); (async () => { const browser = await playwright.webkit.connect({ wsEndpoint: 'wss://cloud.testingbot.com/playwright?key=api_key&secret=api_secret&browserName=safari&browserVersion=latest', }); const context = await browser.newContext(); const page = await context.newPage(); await page.goto('https://testingbot.com/'); await page.screenshot({ path: 'screenshot.png' }); await browser.close(); })(); ``` This example will start a WebKit Browser, navigate to testingbot.com and save a PNG screenshot of the webpage. ## Updating your existing Playwright scripts With a Playwright test, you'll usually start a browser with `await pw.chromium.launch()` To start using our service, simply replace this line with our browser endpoint: ### Before ```javascript const browser = await pw.chromium.launch(); ``` ### After ```javascript await pw.chromium.connect({ wsEndpoint: 'wss://cloud.testingbot.com/playwright?key=api_key&secret=api_secret&browserName=edge&browserVersion=latest', }); ``` ## Specifying browser and version To specify on which browser and version your Playwright test should run, you can include both a `browserName` and `browserVersion` in the `wsEndpoint` URL. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Updating Test Details Every Playwright test will appear in the TestingBot dashboard, by default as an unnamed test. You can specify various parameters in the `wsEndpoint` to customize this. Specify a [name](https://testingbot.com/support/web-automate/playwright/options#name), [extra (metadata) information](https://testingbot.com/support/web-automate/playwright/options#customdata) or group tests together as a [build](https://testingbot.com/support/web-automate/playwright/options#build). TestingBot has created a custom Playwright command, which you can use in your tests, to mark a test as passed or failed in the TestingBot dashboard. See the example below, where we run a Playwright script and depending on whether the test passes or fails, we send back the correct test status to TestingBot. ```javascript const pw = require('playwright-core') const expect = require('chai').expect (async () => { const browser = await pw.chromium.connect({ wsEndpoint: 'wss://cloud.testingbot.com/playwright?key=api_key&secret=api_secret&browserName=chrome&browserVersion=latest', }) const context = await browser.newContext() const page = await context.newPage() await page.goto('https://testingbot.com/') const title = await page.title() try { expect(title).to.equal("Cross Browser Testing and Mobile App Testing. Test on web browsers, mobile emulators, simulators and physical mobile devices.", 'Expected page title is incorrect!') // This is ok, so mark the test as passed await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'setSessionStatus',arguments: {passed: true, reason: 'Title matched'}})}`) } catch { // Test failed, mark the test as failed on TestingBot await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'setSessionStatus',arguments: {passed: false, reason: 'Title did not match'}})}`) } await browser.close() })() ``` ## Integration with Playwright Test Playwright comes with its own test library: [Playwright Test](https://testingbot.com/support/web-automate/playwright/playwright-test). It is developed by Microsoft and offers a robust set of features to quickly write and run automated browser tests. Playwright test comes with a developer-friendly API that greatly simplifies creating tests. You can even [record Playwright tests with its built-in Test Generator](https://testingbot.com/support/web-automate/playwright/recorder) if you don't want to spend time writing the tests yourself. The framework comes with a lot of extra features which promote best practices: - **Page Object Models:** Group common test logic in Page Object Models to avoid code duplication and enhance the structure of your tests. This in turn simplifies test management. - **Interact with Dialogs:** Playwright can interact with dialogs such as `alert`, `confirm`, `prompt` and even `beforeunload` confirmations. - **accessibility testing:** You can use Playwright to test for various accessibility issues on your website. See the [Accessibility Testing documentation](https://playwright.dev/docs/accessibility-testing) for more information. ## Playwright and TestingBot Tunnel Using Playwright in combination with [TestingBot Tunnel](https://testingbot.com/support/tunnel) allows you to test websites behind a firewall or on your local machine with Playwright on TestingBot. ### Setup Tunnel To use TestingBot Tunnel in combination with Playwright, you first need to start the TestingBot Tunnel and pass it a `tunnelIdentifier` name of your liking. ```bash java -jar testingbot-tunnel.jar key secret -i myPlaywrightTunnel ``` In this example, we'll start a TestingBot Tunnel with identifier **myPlaywrightTunnel**. Next, we'll need to configure our Playwright script to use the tunnel, using the same identifier. ### Setup Playwright To let Playwright know which tunnel we want to use, we need to specify the identifier from before as a parameter. Please see the example below where we pass the `tunnelIdentifier` with **myPlaywrightTunnel**. ```javascript const pw = require('playwright-core'); (async () => { const browser = await pw.chromium.connect({ wsEndpoint: 'wss://cloud.testingbot.com/playwright?key=api_key&secret=api_secret&browserName=chrome&browserVersion=latest&tunnelIdentifier=myPlaywrightTunnel', }); const context = await browser.newContext(); const page = await context.newPage(); await page.goto('http://localhost:3000/'); await page.screenshot({ path: 'screenshot.png' }); await browser.close(); })(); ``` This Playwright script will open the latest Chrome browser on the TestingBot browser grid, use the tunnel to access your localhost website and take a screenshot from that page. ### Playwright and TestingBot Tunnel Launcher [testingbot-tunnel-launcher](https://github.com/testingbot/testingbot-tunnel-launcher) is a NodeJS library which allows you to manage the tunnel from within your NodeJS script. To use it, you can install with `npm install testingbot-tunnel-launcher` and use it with Playwright: ```javascript const testingbotTunnel = require('testingbot-tunnel-launcher') const pw = require('playwright-core') ;(async () => { const tunnel = await new Promise((resolve, reject) => { testingbotTunnel({ apiKey: 'api_key', apiSecret: 'api_secret', verbose: true, tunnelIdentifier: 'playwrightTunnel' }, function (err, tunnel) { if (err) { reject(err) return } resolve(tunnel) }) }) const browser = await pw.chromium.connect({ wsEndpoint: 'wss://cloud.testingbot.com/playwright?key=api_key&secret=api_secret&browserName=chrome&browserVersion=latest&tunnelIdentifier=playwrightTunnel', }) const context = await browser.newContext() const page = await context.newPage() await page.goto('http://localhost:8080/') await page.screenshot({ path: 'screenshot.png' }) await browser.close() await tunnel.close() })() ``` This example will start a new Tunnel, start a Playwright script on the latest Chrome version in TestingBot, and take a screenshot of the website running on your local computer `(http://localhost:8080)` ## Playwright Agents Playwright comes with [3 agents](https://playwright.dev/docs/test-agents) that can be used to create tests. - **planner** explores the app and produces a Markdown test plan - **generator** transforms the Markdown plan into the Playwright Test files - **healer** executes the test suite and automatically repairs failing tests, for example due to bad/missing locators To get started, use this command to initialize the agent definitions: ```bash npx playwright init-agents --loop=claude ``` Once the tests are created, you can simply point these to use a remote TestingBot browser by replacing the `launch` command with a `connect` command, as shown in the previous sections. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/playwright/live-view # Test Live View When running automated Playwright tests with TestingBot, you can see a realtime view of the test while it is running. If you are running the tests in a CI/CD pipeline, or on your own machines, it might be useful to see what is happening during the test execution. TestingBot offers a realtime live view of your Playwright tests, so you can see what is happening during the test execution. You can embed this live view in your own dashboards, or open it in a separate window. The live view also supports human in the loop: you can take control of the browser during the test execution, with mouse and keyboard. Realtime Views can be useful for: - Debugging tests by seeing what is happening during the test execution or share a view/control link with colleagues. - Human in the loop: instantly take control or provide input during a test. For example if the test gets stuck waiting for a Captcha or user input (for example authentication). - Embedding in other applications or dashboards for a unified view of test progress. ## Getting started You will need to generate a specific URL to access the live view of your test. To generate the URL, you will need the `sessionId` of the test that is running. To fetch the sessionId in your Playwright script, you can use the custom TestingBot command: ```javascript const testingBotResponse = await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'getSessionDetails'})}`) ``` The response will contain a `sessionId`, which is a unique identifier for this test. Once you have the sessionId, you can generate an authentication token as follows: The authentication hash is created with MD5. Please see the examples below. [NodeJS](https://testingbot.com#) [Java](https://testingbot.com#) [Python](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) ```ruby require 'testingbot' api = TestingBot::Api.new(key, secret) api.get_authentication_hash(sessionId) ``` ```java TestingbotREST restApi = new TestingbotREST(key, secret); String hash = restApi.getAuthenticationHash(String sessionId); ``` ```php $api = new TestingBot\TestingBotAPI($key, $secret); $api->getAuthenticationHash($sessionId); ``` ```python import testingbotclient tb = testingbotclient.TestingBotClient(key, secret) tb.get_share_link(sessionId) ``` ```javascript const TestingBot = require('testingbot-api'); const api = new TestingBot({ api_key: "your-tb-key", api_secret: "your-tb-secret" }); const hash = api.getAuthenticationHashForSharing(sessionId); ``` Now that you have the authentication hash, you can construct the live view URL as follows: ``` https://testingbot.com/tests//live?auth= ``` ## Embed Once you've constructed the live view URL, you can embed it in an iframe within your own application or dashboard: ```html ``` Adjust the width and height attributes as needed to fit your layout. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/playwright/capabilities # Playwright Capabilities Generator Generate Playwright connection options for your automated tests. Choose your browser, device emulation and configuration options to get the exact code you need for your Playwright tests. [![Selenium capabilities](https://testingbot.com/assets/support/selenium-266ecc0599068fc6eca0bcd9174fb46053c95cebe1c66f7a50c480f3f1a52929.svg) Selenium ](https://testingbot.com/support/web-automate/selenium/capabilities) [![Appium capabilities](https://testingbot.com/assets/support/appium-4b35a8bb56bae236bd84c539cee9b8bbb489a55ddf48e32d059c86685d230354.svg) Appium ](https://testingbot.com/support/app-automate/appium/capabilities) [![Puppeteer capabilities](https://testingbot.com/assets/support/puppeteer-0a8cdd38af69dabd46638afbb3bffc8a372902949aa599663ec379f3e875263c.svg) Puppeteer ](https://testingbot.com/support/web-automate/puppeteer/capabilities) [![Playwright capabilities](https://testingbot.com/assets/support/playwright-2dcb4e75b55ac0d1135cfa47be6137b13f46a2047ce78b96a34f51b0e2cc0d96.svg) Playwright ](https://testingbot.com/support/web-automate/playwright/capabilities) Browser Environment ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser Selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. Resolution 800x600 1024x768 1280x800 1280x960 1280x1024 1440x900 1680x1050 1600x1200 1920x1200 1920x1080 2560x1440 ## Project Capabilities Test Name Allows you to customize the name of the test, as it will appear in the dashboard. Build Name Group tests under the same build identifier for easy retrieval. ## Debugging Record Video Yes No ## Customization Headless Mode Run the browser without rendering UI for faster test execution. Yes No Geolocation Testing Run the test from a different geographical location. None Random Country Australia Belgium Brazil Canada Chile France Germany India Italy Japan Norway Singapore South Africa Sweden Switzerland United Arab Emirates United Kingdom United States Timezone ([list of timezones](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones)) Prerun Script ([documentation](https://testingbot.com/support/web-automate/selenium/test-options#prerun)) Upload URLs Looking for more options? See [all Playwright capabilities](https://testingbot.com/support/web-automate/playwright/options). ```javascript const playwright = require('playwright-core'); (async () => { const capabilities = { browserName: 'chrome', browserVersion: 'latest', platformName: 'Windows 11', 'tb:options': { 'key': process.env.TB_KEY, 'secret': process.env.TB_SECRET } } const browser = await playwright.chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }); const context = await browser.newContext(); const page = await context.newPage(); await page.goto('https://testingbot.com/'); await page.screenshot({ path: 'screenshot.png' }); await browser.close(); })(); ``` --- URL: https://testingbot.com/support/web-automate/playwright/playwright-test # Playwright Test Playwright comes with its own test-runner called Playwright Test. It is designed to make end-to-end testing easy with Playwright. More information about `@playwright/test` is available in the [Playwright documentation](https://playwright.dev/docs/test-configuration). See the TestingBot [Playwright Test Node Example repository](https://github.com/testingbot/node-playwright-test-example) for a simple example on how to run Playwright tests with TestingBot. ## Installation To get started, please install the Playwright Test package: ```bash npm i -D @playwright/test ``` ## Configuration Next, we will need to create a [configuration file](https://playwright.dev/docs/test-configuration) so that Playwright Test knows what tests to run and what options to use. Create a file called `playwright.config.js` with the following contents: ```javascript import { defineConfig } from '@playwright/test' import { getConnectWsEndpoint } from './testingbot.config' /** * See https://playwright.dev/docs/test-configuration. */ export default defineConfig({ testDir: './tests', /* Maximum time one test can run for. */ timeout: 30 * 1000, /* Fail the build on CI if you accidentally left test.only in the source code. */ forbidOnly: !!process.env.CI, /* Retry on CI only */ retries: process.env.CI ? 2 : 0, /* Opt out of parallel tests on CI. */ workers: process.env.CI ? 1 : undefined, /* Reporter to use. See https://playwright.dev/docs/test-reporters */ reporter: 'html', /* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */ use: { /* Base URL to use in actions like `await page.goto('/')`. */ // baseURL: 'http://localhost:3000', /* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */ trace: 'on-first-retry', }, projects: [ { name: 'playwright-chrome@latest:Windows 10', use: { connectOptions: { wsEndpoint: getConnectWsEndpoint({ browserName: 'chrome', browserVersion: 'latest', platform: 'WIN10' }) } }, }, { name: 'playwright-firefox@latest:Linux', use: { connectOptions: { wsEndpoint: getConnectWsEndpoint({ browserName: 'firefox', browserVersion: 'latest', platform: 'LINUX' }) } } } ] }) ``` You will also need to create a `testingbot.config.js` file, which will contain code to connect to our Playwright server: ```javascript export function getConnectWsEndpoint(userCapabilities) { const defaultCapabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET } } const capabilities = { ...defaultCapabilities, ...userCapabilities } const connectUrl = `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` return connectUrl } ``` You're now all set up to run your first Playwright Test on TestingBot. Let's see the [example](https://testingbot.com#example) below to run a test. ## Specifying browser and version To specify on which browser and version your Playwright test should run, you can include both a `browserName` and `browserVersion` in the `wsEndpoint` URL. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Example Test As we've [previously defined in our playwright.config.js file](https://testingbot.com#configuration), our tests should be placed in the `tests` directory. Create a new file in the `tests` directory called `sample.spec.js`: ```javascript const { test, expect } = require('@playwright/test') test('Sample TestingBot test', async ({ page }, testInfo) => { try { await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'setSessionName', arguments: { name: testInfo.project.name }})}`) await page.goto('https://testingbot.com/',{ waitUntil: 'networkidle' }) await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'setSessionStatus', arguments: { passed: true }})}`) } catch (e) { await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'setSessionStatus', arguments: { passed: false, reason: e.message }})}`) } }) ``` To run the test, execute this command in the directory where you placed the configuration file: ```bash npx playwright test ``` This test will set the correct name to be shown in the TestingBot dashboard, navigate with the remote browser to the TestingBot homepage and then set the test as passed. ## Reports There are various [Test Reporters](https://playwright.dev/docs/test-reporters) that Playwright offers, which you can change in the [configuration file](https://testingbot.com#configuration). For every test that runs on TestingBot, you will receive a video of the test, generated logs and other meta-data. ## Other Options You can set various [Timeout options](https://playwright.dev/docs/test-timeouts), use [annotations](https://playwright.dev/docs/test-annotations) and set up [setup and teardown scripts](https://playwright.dev/docs/test-global-setup-teardown). The global setup and teardown scripts can be used to start (and stop) a [TestingBot Tunnel](https://testingbot.com/support/tunnel), which allows you to test on websites that are not publicly available. An example of a setup script: ```javascript const testingbotTunnel = require('testingbot-tunnel-launcher') module.exports = async () => { await new Promise((resolve, reject) => { testingbotTunnel({ apiKey: process.env.TB_KEY, apiSecret: process.env.TB_SECRET, verbose: true }, function (err, tunnel) { if (err) { console.error(err.message) return reject(err) } resolve() }) }) } ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/playwright/visual-regression-testing # Playwright Visual Regression Testing Playwright Test supports [visual comparisons](https://playwright.dev/docs/test-snapshots), where it will take a screenshot of a page and then compare it with a 'golden image'. A golden image is a snapshot of what the correct visual layout should be. Once you have a snapshot, your test can check if the layout still matches the one saved in the snapshot. If it does not match, your test will fail. To get started, let's create a test called `playwright_test.spec.js` with the following contents: ```javascript const { test, expect } = require('@playwright/test') test('should add an item', async ({ page }) => { await page.goto('https://testingbot.com') const title = await page.title() await expect(page).toHaveScreenshot() }) ``` Next, make sure you've created a [playwright.config.js configuration file](https://testingbot.com/support/web-automate/playwright/playwright-test#configuration), which contains details such as: - How to connect to the TestingBot browser grid - On which remote browsers this visual test should run To start the test, please run this command: ```bash npx playwright test ``` The first run will fail, because there's no 'golden image' yet. When you run the test again, Playwright will take a new screenshot and compare it with the golden snapshot. You can [configure the threshold](https://github.com/mapbox/pixelmatch#api), `maxDiffPixels` to your liking: ```javascript await expect(page).toHaveScreenshot({ maxDiffPixels: 100 }) ``` If you build your UI components with Storybook, see [Storybook testing with Playwright](https://testingbot.com/support/web-automate/playwright/storybook) for how to run a visual comparison for every story across browsers and real devices. ## Masked Screenshots Playwright provides the option to mask any specific locator when taking a screenshot. This is useful if you want to do regression testing, and you know that a specific part of the page has content that might cause the test to fail or is not important for your test. For example an ad, random list, or anything else that should not impact your visual comparison test. Simply find one or more locators of elements you want to hide: ```javascript const mask_locator = page.locator("#random-image") ``` Finally, take a screenshot but mask the locators: ```javascript await page.screenshot({ path: 'masked.png', mask: [mask_locator] }) ``` Below is an example of a screenshot with a masked locator: ![Playwright masked screenshot](https://testingbot.com/assets/support/playwright/masked-166e5ef9ecd63165a953ed564e770afd37b47ff4e280f7e9ba12665244af9c81.png) ## Text and binary snapshots Next to comparing images, Playwright also supports comparing text or other (binary) data, through snapshots. The example below shows how to compare the text of a DOM element to make sure it matches its snapshot. ```javascript const { test, expect } = require('@playwright/test') test('example test', async ({ page }) => { await page.goto('https://testingbot.com') expect(await page.textContent('body > div.main > div.hero.home > div > div > p')).toMatchSnapshot('hero.txt') }) ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/playwright/storybook # Storybook Testing [Storybook](https://storybook.js.org/) is a workshop for building UI components in isolation. Each component state is described as a **story** , and every story is reachable at its own URL: ```bash https://your-storybook.example.com/iframe.html?id=button--primary&viewMode=story ``` Because stories are plain URLs, you can open them in any browser or device on the TestingBot cloud. This page shows three ways to test your Storybook components with TestingBot: - **Cross browser screenshot testing** : run a Playwright screenshot test for every story on multiple browsers and browser versions. - **Storybook Test Runner** : point `@storybook/test-runner` at the TestingBot browser grid. - **Real device testing** : render stories in Mobile Safari and Chrome on real iPhones and Android devices. See our [Storybook example repository](https://github.com/testingbot/storybook-testingbot-example) for a runnable version of everything on this page, including a GitHub Actions workflow that builds Storybook, opens a tunnel and screenshots every story on the grid. ## Prerequisites The browsers and devices in the TestingBot cloud need to be able to reach your Storybook. You have two options: - **Publish a static build** : run `npm run build-storybook` and host the resulting `storybook-static` directory on any static host (S3, Netlify, Vercel, GitHub Pages or your own server). - **Use TestingBot Tunnel** : keep Storybook running locally on `localhost:6006` and connect it to the TestingBot cloud with [TestingBot Tunnel](https://testingbot.com/support/tunnel). See [testing a local Storybook](https://testingbot.com#tunnel) below. Storybook publishes a machine readable list of all your stories at `/index.json`. Verify it is available: ```bash curl https://your-storybook.example.com/index.json ``` For Storybook versions older than 9, you may need to enable this file with `buildStoriesJson: true` in `.storybook/main.js`. ## Screenshot testing every story The most flexible approach is a small [Playwright Test](https://testingbot.com/support/web-automate/playwright/playwright-test) suite that reads `index.json`, generates one test per story and compares a screenshot of each story against a golden image with [`toHaveScreenshot`](https://testingbot.com/support/web-automate/playwright/visual-regression-testing). Create a file called `storybook.spec.mjs`: ```javascript import { test, expect } from '@playwright/test' const STORYBOOK_URL = process.env.STORYBOOK_URL || 'https://your-storybook.example.com' // Fetch the list of stories from Storybook's index.json const response = await fetch(`${STORYBOOK_URL}/index.json`) const index = await response.json() const stories = Object.values(index.entries).filter((entry) => entry.type === 'story') for (const story of stories) { test(`story ${story.id}`, async ({ page }) => { await page.goto(`${STORYBOOK_URL}/iframe.html?id=${story.id}&viewMode=story`) // Wait until the story has rendered await page.waitForSelector('#storybook-root') await page.evaluate(() => document.fonts.ready) await expect(page).toHaveScreenshot(`${story.id}.png`, { animations: 'disabled', fullPage: true }) }) } ``` Next, configure Playwright Test to run this suite on the TestingBot grid. Each project in `playwright.config.js` becomes one browser and version combination, so a single run gives you screenshots of every story on every browser you care about: ```javascript import { defineConfig } from '@playwright/test' import { getConnectWsEndpoint } from './testingbot.config' export default defineConfig({ testDir: './tests', timeout: 60 * 1000, projects: [ { name: 'chrome@latest:Windows', use: { connectOptions: { wsEndpoint: getConnectWsEndpoint({ browserName: 'chrome', browserVersion: 'latest', platform: 'WIN10' }) } } }, { name: 'firefox@latest:Linux', use: { connectOptions: { wsEndpoint: getConnectWsEndpoint({ browserName: 'firefox', browserVersion: 'latest', platform: 'LINUX' }) } } }, { name: 'safari@latest:macOS', use: { connectOptions: { wsEndpoint: getConnectWsEndpoint({ browserName: 'safari', browserVersion: 'latest', platform: 'SONOMA' }) } } } ] }) ``` The `getConnectWsEndpoint` helper is described in our [Playwright Test configuration guide](https://testingbot.com/support/web-automate/playwright/playwright-test#configuration): ```javascript export function getConnectWsEndpoint(userCapabilities) { const defaultCapabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET } } const capabilities = { ...defaultCapabilities, ...userCapabilities } return `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` } ``` Run the suite: ```bash npx playwright test ``` The first run creates the golden images. Subsequent runs compare each story against its golden image and fail when a component's rendering changes. You can pin specific [browser versions](https://testingbot.com/support/web-automate/playwright/browsers) to catch regressions that only appear on the versions your users run, and use Playwright's built-in parallelism to spread stories over multiple [parallel sessions](https://testingbot.com/support/web-automate/playwright/playwright-parallel-testing). Every session appears in your [TestingBot dashboard](https://testingbot.com/members) with video, screenshots and logs. ## Storybook Test Runner If you already use the [Storybook Test Runner](https://storybook.js.org/docs/writing-tests/integrations/test-runner) (`@storybook/test-runner`) to execute your play functions, you can point it at the TestingBot grid. The test runner drives Jest through a bundled `jest-playwright` preset, and that preset connects to an existing browser instead of launching a local one when you give it a `wsEndpoint`. First, eject the test runner's Jest configuration: ```bash npx test-storybook --eject ``` That writes a `test-runner-jest.config.js` file. Add a `connectOptions.wsEndpoint` to it: ```javascript import { getJestConfig } from '@storybook/test-runner' const testRunnerConfig = getJestConfig() const capabilities = { browserName: 'chrome', browserVersion: 'latest', platform: 'WIN11', 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET } } export default { ...testRunnerConfig, testEnvironmentOptions: { ...testRunnerConfig.testEnvironmentOptions, 'jest-playwright': { ...testRunnerConfig.testEnvironmentOptions['jest-playwright'], browsers: ['chromium'], connectOptions: { wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` } } } } ``` Finally, run the test runner against a Storybook URL that the remote browser can reach: ```bash npx test-storybook --url https://your-storybook.example.com ``` Each story is visited on the remote TestingBot browser, play functions are executed and assertion failures are reported by Jest as usual. The connect settings have to go in `testEnvironmentOptions` as shown above. A standalone `jest-playwright.config.js` file is ignored, because the test runner's default Jest config always defines `testEnvironmentOptions['jest-playwright']` and the bundled preset skips its config file whenever that key is present. Older guides that use a `connectBrowserApp` key describe an earlier `jest-playwright` API that current releases no longer read. The `jest-playwright` project has been deprecated by its maintainers and Storybook is migrating the test runner to `@playwright/test`. Because the configuration above depends on internals of the bundled preset, pin the `@storybook/test-runner` version you tested. For a long-term setup we recommend the [Playwright Test approach](https://testingbot.com#cross-browser) described earlier on this page. ## Stories on real mobile devices Emulated viewports show you a small screen, but not how Mobile Safari or Chrome on Android actually render your component. Since every story is a URL, you can open it on [real mobile devices](https://testingbot.com/real-device-testing) in the TestingBot cloud. ### Playwright on real Android devices Playwright tests can run against Chrome on real Android devices in our cloud. Add a `deviceName` and `realDevice: true` to your capabilities, as described in the [Playwright mobile testing guide](https://testingbot.com/support/web-automate/playwright/mobile). The screenshot suite from the [previous section](https://testingbot.com#cross-browser) works unchanged; add a project with real device capabilities to your `playwright.config.js`. ### Mobile Safari and Chrome through Appium Playwright cannot drive browsers on real iOS devices. To see your stories in Mobile Safari on a real iPhone or iPad, use a WebDriver session, for example with [WebdriverIO](https://testingbot.com/support/web-automate/selenium/nodejs/webdriverio): ```javascript const { remote } = require('webdriverio') const STORYBOOK_URL = 'https://your-storybook.example.com' ;(async () => { const browser = await remote({ hostname: 'hub.testingbot.com', capabilities: { browserName: 'safari', platformName: 'iOS', browserVersion: '18.0', 'appium:deviceName': 'iPhone 15', 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, realDevice: true, name: 'Storybook on real iPhone' } } }) const response = await fetch(`${STORYBOOK_URL}/index.json`) const index = await response.json() const stories = Object.values(index.entries).filter((entry) => entry.type === 'story') for (const story of stories) { await browser.url(`${STORYBOOK_URL}/iframe.html?id=${story.id}&viewMode=story`) await browser.$('#storybook-root').waitForExist() await browser.saveScreenshot(`./screenshots/${story.id}.png`) } await browser.deleteSession() })() ``` Use `browserName: 'chrome'` with `platformName: 'Android'` and an Android `deviceName` to run the same script on a real Android device. See the [device list](https://testingbot.com/support/app-automate/devices) for available devices, and pin both the device name and the version, since the same model is offered on several OS versions. Real devices cannot resolve the hostname `localhost`, so a tunnelled Storybook needs a real hostname for this tier. Add an entry such as `127.0.0.1 storybook.local` to the `/etc/hosts` file on the machine running the tunnel and use `http://storybook.local:6006` as your Storybook URL. Publishing a static build avoids the problem entirely. To compare these device screenshots against baselines and review differences in a dashboard, you can feed them into [TestingBot Visual Testing](https://testingbot.com/support/visual-testing). ## Testing a local Storybook During development, Storybook runs on `localhost:6006`, which the TestingBot cloud cannot reach directly. Start a [TestingBot Tunnel](https://testingbot.com/support/tunnel) to make it available: ```bash java -jar testingbot-tunnel.jar key secret -i myStorybookTunnel ``` Then add the `tunnelIdentifier` to your capabilities and point your tests at `http://localhost:6006`. Storybook's default port needs one extra capability, `localHttpPorts`, explained below: ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, tunnelIdentifier: 'myStorybookTunnel', localHttpPorts: [6006] }, browserName: 'chrome', browserVersion: 'latest' } ``` The tunnel forwards the ports `80`, `443`, `3000`, `3001`, `3030`, `3400` and `8080` by default. Storybook's default port `6006` is not one of them, so it has to be requested with [`localHttpPorts`](https://testingbot.com/support/web-automate/selenium/test-options#localhost). Without it the tunnel starts fine and the session starts fine, but every story fails to load, which looks exactly like a broken tunnel. If you serve Storybook on a port that is already forwarded, such as `3000`, you do not need this capability. The remote browser will now access your local Storybook through the tunnel. ## Frequently asked questions How do I run Storybook tests on TestingBot? Every Storybook story is reachable at a URL: iframe.html?id=&viewMode=story. Enumerate your stories from index.json, then use Playwright's connectOptions to open each story URL on a TestingBot browser and take a screenshot with toHaveScreenshot. Your Storybook needs to be reachable by the remote browser, either as a published static build or through TestingBot Tunnel. Can I use the Storybook Test Runner with TestingBot? Yes. The Storybook Test Runner (@storybook/test-runner) is built on Jest and Playwright. Eject its configuration with test-storybook --eject, then add a connectOptions.wsEndpoint pointing to wss://cloud.testingbot.com/playwright inside testEnvironmentOptions['jest-playwright'] in the generated test-runner-jest.config.js. Each story is then visited on a remote TestingBot browser and its play function runs there. Can I test Storybook components on real mobile devices? Yes. Because each story is a URL, you can open it in Mobile Safari or Chrome on real iOS and Android devices through an Appium session, or with [Playwright on real Android devices](https://testingbot.com/support/web-automate/playwright/mobile), and capture screenshots of how the component renders on real hardware. Do I need to install browsers in CI to test Storybook on TestingBot? No. When you connect to the TestingBot grid, the browsers run in our cloud. You only need playwright-core in your CI image, no npx playwright install step. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/playwright/performance # Measuring Performance with Playwright Today, it is more important than ever to make sure your website loads fast. The speed of your website matters for a lot of reasons: - Faster websites receive more organic search engine traffic - Improved user experiences cause an increase in online sales - Less frustration when using your product Making your webpages faster takes a lot of effort and requires measurements to figure out what to optimise and how to optimise. Modern browsers expose a lot of [performance metrics](https://web.dev/metrics/) that can help you with determining what to improve and which changes to make. Playwright can help with identifying reasons that cause slowdowns, by querying the various performance related APIs that browsers provide. Chromium browsers for example provide a rich set of metrics, called [Web Vitals metrics](https://web.dev/vitals/). These metrics include Time to First Byte (TTFB), Total Blocking Time (TBT) and First Contentful Paint (FCP), which are good user experience indicators and worth monitoring in your tests. ## Navigation and Resource Timing API The [Navigation Timing API](https://developer.mozilla.org/en-US/docs/Web/API/Navigation_timing_API) allows you to retrieve all events and timings related to pageload times. The response contains various interesting fields, such as `domComplete` time, `duration` and `connectEnd`. You can use these metrics in your tests, to detect performance regressions for each webpage you are testing. ### Performance Budget Your Playwright tests could use a performance budget, where you want to make sure that the performance of a webpage does not go below a specific standard that you set. For example, see the following Playwright code where we detect the `domComplete`, if it goes below a threshold we want to fail the testcase, alerting us of a possible performance regression. ```javascript const pw = require('playwright-core'); (async () => { const browser = await pw.chromium.connect({ wsEndpoint: 'wss://cloud.testingbot.com/playwright?key=api_key&secret=api_secret&browserName=chrome&browserVersion=latest', }) const context = await browser.newContext() const page = await context.newPage() await page.goto('https://testingbot.com/'); const navigationTimingJson = await page.evaluate(() => JSON.stringify(performance.getEntriesByType('navigation')) ) const navigationTiming = JSON.parse(navigationTimingJson) if (navigationTiming[0].domComplete > 2000) { console.error('ERROR: domComplete above 2 seconds') } await browser.close() })() ``` ### Resource Timing API The [Resource Timing API](https://developer.mozilla.org/en-US/docs/Web/API/Performance_API/Resource_timing) provides a way to get timing information for specific resources on the page. For example, you can check how quickly an image loaded on your page. ```javascript const pw = require('playwright-core'); (async () => { const browser = await pw.chromium.connect({ wsEndpoint: 'wss://cloud.testingbot.com/playwright?key=api_key&secret=api_secret&browserName=chrome&browserVersion=latest', }) const context = await browser.newContext() const page = await context.newPage() await page.goto('https://testingbot.com/'); const resourceTimingJson = await page.evaluate(() => JSON.stringify(window.performance.getEntriesByType('resource')) ) const resourceTiming = JSON.parse(resourceTimingJson) const logoResourceTiming = resourceTiming.find((element) => element.name.includes('.webp') ) console.log(logoResourceTiming) await browser.close() })() ``` ## Paint Timing API You can use the [Paint Timing API](https://developer.mozilla.org/en-US/docs/Web/API/PerformancePaintTiming) to retrieve timings on First Paint (FP) and First Contentful Paint (FCP). ```javascript const pw = require('playwright-core'); (async () => { const browser = await pw.chromium.connect({ wsEndpoint: 'wss://cloud.testingbot.com/playwright?key=api_key&secret=api_secret&browserName=chrome&browserVersion=latest', }) const context = await browser.newContext() const page = await context.newPage() await page.goto('https://testingbot.com/'); const paintTimingJson = await page.evaluate(() => JSON.stringify(window.performance.getEntriesByType('paint')) ) const paintTiming = JSON.parse(paintTimingJson) console.log(paintTiming) await browser.close() })() ``` ## Layout Instability API The Layout Instability API provides information regarding layout shifts. Layout shifts happen when the layout of your webpage changes during the loading of the page. You can use this API to evaluate the [Core Web Vital Cumulative Layout Shift](https://web.dev/cls/) (CLS). ```javascript const pw = require('playwright-core'); (async () => { const browser = await pw.chromium.connect({ wsEndpoint: 'wss://cloud.testingbot.com/playwright?key=api_key&secret=api_secret&browserName=chrome&browserVersion=latest', }) const context = await browser.newContext() const page = await context.newPage() await page.goto('https://testingbot.com/'); const cumulativeLayoutShift = await page.evaluate(() => { return new Promise((resolve) => { let CLS = 0 new PerformanceObserver((l) => { const entries = l.getEntries() entries.forEach(entry => { if (!entry.hadRecentInput) { CLS += entry.value } }) resolve(CLS) }).observe({ type: 'layout-shift', buffered: true }) }) }) console.log(parseFloat(cumulativeLayoutShift)) await browser.close() })() ``` ## Long Task API The [Long Task API](https://developer.mozilla.org/en-US/docs/Web/API/Long_Tasks_API) returns a list of Javascript executions that take more than 50 milliseconds to complete (block the UI thread). You can use this API to calculate the [Total Blocking Time](https://web.dev/tbt/) (TBT). To calculate the TBT value, we need to use a `PerformanceObserver`. This will observe longtasks entries and make a sum of the differences to the maximal JavaScript execution time of 50 milliseconds. ```javascript const pw = require('playwright-core'); (async () => { const browser = await pw.chromium.connect({ wsEndpoint: 'wss://cloud.testingbot.com/playwright?key=api_key&secret=api_secret&browserName=chrome&browserVersion=latest', }) const context = await browser.newContext() const page = await context.newPage() await page.goto('https://testingbot.com/'); const totalBlockingTime = await page.evaluate(() => { return new Promise((resolve) => { let totalBlockingTime = 0 new PerformanceObserver(function (list) { const perfEntries = list.getEntries() for (const perfEntry of perfEntries) { totalBlockingTime += perfEntry.duration - 50 } resolve(totalBlockingTime) }).observe({ type: 'longtask', buffered: true }) // Resolve promise if there haven't been any long tasks setTimeout(() => resolve(totalBlockingTime), 5000) }) }) console.log(parseFloat(totalBlockingTime)) await browser.close() })() ``` ## Chrome DevTools Performance The Chrome DevTools Protocol provides many great performance tools that you can use. You can check the [DevTools Protocol documentation](https://chromedevtools.github.io/devtools-protocol/) for an extensive overview of all the possibilities. One of the examples you can use with DevTools is throttling the network. To use this, you need to send the `Network.emulateNetworkConditions` command. The syntax to connect to TestingBot is slightly different, as we now need to use `connectOverCDP`. ```javascript const pw = require('playwright-core'); (async () => { const browser = await pw.chromium.connectOverCDP({ wsEndpoint: 'wss://cloud.testingbot.com/playwright?key=api_key&secret=api_secret&browserName=chrome&browserVersion=latest', }) const context = browser.contexts()[0] const page = context.pages()[0] const client = await context.newCDPSession(page) await client.send('Network.enable') await client.send('Network.emulateNetworkConditions', { offline: false, downloadThroughput: (5 * 1024 * 1024) / 8, uploadThroughput: (4 * 1024 * 1024) / 8, latency: 30 }) await page.goto('https://testingbot.com/'); await browser.close() })() ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/playwright/pytest # Playwright & PyTest To use PyTest with Playwright, we'll need a Python library that automates via Playwright. Microsoft's [playwright-python](https://github.com/microsoft/playwright-python) library provides a Python library which does Automation via Playwright. To get started, please install these packages: ```bash pip install pytest-playwright playwright install ``` Now you're ready to use Python with Playwright. You can take a look at the [playwright-python examples](https://github.com/microsoft/playwright-python#usage) on how to do this. ## Example To get started, please see this simple example below. This will start a new Chrome browser in the TestingBot browser grid, and allow PyTest to control the browser via Playwright. ```python import pytest from playwright.sync_api import Playwright @pytest.fixture def browser(playwright: Playwright): return playwright.chromium.connect(ws_endpoint = 'wss://cloud.testingbot.com/playwright?key=api_key&secret=api_secret&browserName=chrome&browserVersion=latest', timeout=90000) def test_title(browser): page = browser.new_page() page.goto('https://testingbot.com') assert page.title() == 'TestingBot: Cross Browser Testing and Mobile App Testing' browser.close() ``` This example will open a Chrome browser, navigate to the TestingBot homepage and verify the page's title. ## Parallel Testing with PyTest One of the great advantages of our service is that you can run multiple tests simultaneously (in parallel). This drastically shortens the total duration of your test suite, as multiple tests will run concurrently. We recommend using [pytest-parallel](https://pypi.org/project/pytest-parallel/). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/playwright/robotframework # Robot Framework & Playwright [Browser Library](https://robotframework-browser.org/) allows you to run Playwright tests with Robot Framework. By using TestingBot, you will be able to run Robot tests using Playwright on TestingBot's browser grid. The advantage is that you do not have to set up and maintain a fleet of different browsers, versions and operating systems. Another advantage is that you can run multiple tests, in parallel, on TestingBot's grid. ## Example Below is an example on how to connect to the TestingBot browser grid, using Browser Library's [Connect To Browser](https://marketsquare.github.io/robotframework-browser/Browser.html#Connect%20To%20Browser). To run the test, simply save it in a file (for example `test.robot`) and call `python -m robot test.robot` from the command line. ```python ***Settings*** Library Browser Test Setup Setup TestingBot ***Variables*** &{desiredCapabilities} platform=WIN10 ... browserName=chrome ... key=add-your-TestingBot-key ... secret=add-your-TestingBot-secret ***Keywords*** Setup TestingBot ${TBCAPS} Evaluate urllib.parse.quote( json.dumps(${desiredCapabilities}) ) Connect To Browser wsEndpoint=wss://cloud.testingbot.com/playwright?capabilities=${TBCAPS} ***Test Cases*** Playwright Test New Page https://playwright.dev/ Get Title matches Playwright Get Attribute "Get started" href == /docs/intro Click "Get started" Get Url matches .*intro ``` This example will open a Chrome browser using Playwright on TestingBot, navigate to the Playwright website and do some verifications. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/playwright/jest # Playwright & Jest Jest-Playwright allows you to run tests with Jest on browsers controlled with Playwright. The `jest-playwright-preset` package has been deprecated by its maintainers, who recommend migrating to Playwright's own test runner. If you are starting a new project, use [Playwright Test](https://testingbot.com/support/web-automate/playwright/playwright-test) instead. This page is kept for existing Jest suites. To get started, please install the package. `jest-playwright-preset` requires Jest 29, so pin it rather than taking the latest Jest: ```bash npm init && npm install -D jest@29 jest-playwright-preset playwright ``` Next, specify these settings in your Jest configuration file (`jest.config.js`): ```javascript module.exports = { rootDir: '.', testTimeout: 20000, testMatch: [ '/*.spec.js' ], preset: 'jest-playwright-preset' } ``` ## Configure Jest-Playwright To configure Jest Playwright, please create a new file called `jest-playwright.config.js` and add this to the file: ```javascript module.exports = { connectOptions: { wsEndpoint: 'wss://cloud.testingbot.com/playwright?key=api_key&secret=api_secret&browserName=edge&browserVersion=latest' } } ``` This instructs Jest Playwright to connect to the TestingBot browser grid and use the latest Edge browser. Without a `wsEndpoint`, the preset launches a browser on your own machine instead. If your `package.json` sets `"type": "module"`, name the file `jest-playwright.config.cjs` instead. The preset picks the extension based on your package type. You can also point at any path with the `JEST_PLAYWRIGHT_CONFIG` environment variable. ## Run your first test To create a test, create a new file called `sample.spec.js` and add this to the file: ```javascript describe('Google', () => { beforeAll(async () => { await page.goto('https://google.com') }) it('should display google text on page', async () => { await expect(page).toMatch('google') }) }) ``` Now you can run your first test with Playwright on the TestingBot cloud: ```bash jest ``` This will open Google in an Edge browser and verify if the word 'google' is on the page. You should see something like this in your terminal after running the command: ``` PASS ./sample.spec.js (6.503s) Google ✓ should display google text on page (81ms) Test Suites: 1 passed, 1 total Tests: 1 passed, 1 total ``` This test will also appear in your [Member Dashboard](https://testingbot.com/members). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/playwright/mocha # Playwright & Mocha [Mocha](https://mochajs.org/) is a popular Javascript/NodeJS test runner, which is great for E2E testing. See the example below on how to use Mocha and Playwright to run tests on the TestingBot browser grid. ## Installation To get started, please install these packages: ```bash npm i mocha chai playwright-core ``` Now you can create a basic test script. Usually, you'll create these in a `test` folder, with a filename ending in `.spec.js`: ```javascript const pw = require('playwright-core') const chai = require('chai') const expect = chai.expect let page, browser, context describe('Sample Test', () => { beforeEach(async function() { this.timeout(35000) const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET }, browserName: 'chrome', browserVersion: 'latest' } browser = await pw.chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) context = await browser.newContext() page = await context.newPage() }) afterEach(async function() { await page.screenshot({ path: `${this.currentTest.title.replace(/\s+/g, '_')}.png` }) await browser.close() }) it('Checks the title of the page', async() => { await page.goto('https://testingbot.com/'); const title = await page.title() expect(title).to.equal('TestingBot: Cross Browser Testing and Mobile App Testing') }) }) ``` This example will use Playwright to connect to a Chrome browser in the TestingBot cloud. It will open the TestingBot website, retrieve the title of the page and verify with `expect` if the title is correct. Chai is used to do assertions, this is a popular assertion library, allowing you to use either `expect`, `should` and `assert`, depending on your preferences. After each test, we'll take a screenshot and close the browser (see the `afterEach` part of the example code). ## Parallel Testing with Mocha One of the great advantages of the TestingBot service is that you can run multiple tests simultaneously. This drastically shortens the total duration of your test suite, as multiple tests will run concurrently. By default, Mocha does not run multiple test files in parallel. To achieve parallelism, you can use `mocha-parallel-tests`, which is an NPM package that will run each of your test files in a separate process. ```bash npm i mocha-parallel-tests ``` Now you can create multiple `test/*.spec.js` files with different tests: `test/test-one.spec.js`: ```javascript const pw = require('playwright-core') const chai = require('chai') const expect = chai.expect let page, browser, context describe('Sample Test', () => { beforeEach(async function() { this.timeout(35000) const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET }, browserName: 'chrome', browserVersion: 'latest' } browser = await pw.chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) context = await browser.newContext() page = await context.newPage() }) afterEach(async function() { await page.screenshot({ path: `${this.currentTest.title.replace(/\s+/g, '_')}.png` }) await browser.close() }) it('Checks the title of the page', async() => { await page.goto('https://testingbot.com/'); const title = await page.title() expect(title).to.equal('TestingBot: Cross Browser Testing and Mobile App Testing') }) }) ``` `test/test-two.spec.js`: ```javascript const pw = require('playwright-core') const chai = require('chai') const expect = chai.expect let page, browser, context describe('Sample Test', () => { beforeEach(async function() { this.timeout(35000) const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET }, browserName: 'chrome', browserVersion: 'latest' } browser = await pw.chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) context = await browser.newContext() page = await context.newPage() }) afterEach(async function() { await page.screenshot({ path: `${this.currentTest.title.replace(/\s+/g, '_')}.png` }) await browser.close() }) it('Checks the title of the page', async() => { await page.goto('https://testingbot.com/pricing'); const title = await page.title() expect(title).to.equal('Cross Browser Testing - Prices and plans available for test online.') }) }) ``` To run these 2 test files simultaneously, please run this command: ```bash mocha-parallel-tests test/*.spec.js ``` You should see output similar to: ```bash Sample Test ✓ Checks the title of the page (798ms) Sample Test ✓ Checks the title of the page (921ms) 2 passing (26s) ``` Depending on how many tests you run in parallel, you can drastically shorten your total test duration, and get faster feedback from your tests. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/playwright/java # Playwright & Java Playwright for Java provides a powerful API for browser automation that can be used for end-to-end testing. See the example below on how to use Java with Playwright to run tests on the TestingBot browser grid. More information about Playwright for Java can be found on the [Playwright Java documentation](https://playwright.dev/java/docs/intro). ## Installation To get started, add the Playwright dependency to your Maven project: ```xml com.microsoft.playwright playwright 1.53.0 ``` Or if you're using Gradle: ```groovy implementation 'com.microsoft.playwright:playwright:1.53.0' ``` Now you can create a basic test script. Below is an example using JUnit 5: ```java package com.example; import com.google.gson.JsonObject; import com.microsoft.playwright.*; import org.junit.jupiter.api.*; import static org.junit.jupiter.api.Assertions.assertEquals; class PlaywrightTest { static Playwright playwright; static Browser browser; Page page; @BeforeAll static void launchBrowser() { // Create the capabilities object with authentication information JsonObject capabilities = new JsonObject(); JsonObject tbOptions = new JsonObject(); tbOptions.addProperty("key", "YOUR_KEY"); tbOptions.addProperty("secret", "YOUR_SECRET"); capabilities.add("tb:options", tbOptions); capabilities.addProperty("browserName", "chrome"); capabilities.addProperty("browserVersion", "latest"); capabilities.addProperty("platform", "WIN10"); // Create a custom browser type to connect to TestingBot playwright = Playwright.create(); String wsEndpoint = "wss://cloud.testingbot.com/playwright?capabilities=" + java.net.URLEncoder.encode(capabilities.toString(), java.nio.charset.StandardCharsets.UTF_8); browser = playwright.chromium().connect(new BrowserType.ConnectOptions() .setWsEndpoint(wsEndpoint)); } @AfterAll static void closeBrowser() { if (playwright != null) { playwright.close(); playwright = null; } } @BeforeEach void createContextAndPage() { BrowserContext context = browser.newContext(); page = context.newPage(); } @AfterEach void closeContext() { if (page != null) { // Take a screenshot after the test page.screenshot(new Page.ScreenshotOptions() .setPath(java.nio.file.Paths.get("screenshot-" + java.util.UUID.randomUUID() + ".png"))); page.context().close(); page = null; } } @Test void shouldCheckPageTitle() { page.navigate("https://testingbot.com/"); String title = page.title(); assertEquals("TestingBot: Cross Browser Testing and Mobile App Testing", title); } } ``` This example will use Playwright to connect to a Chrome browser in the TestingBot cloud. It will open the TestingBot website, retrieve the title of the page and verify with JUnit assertions if the title is correct. You can also use test frameworks like TestNG with Playwright, following a similar approach. Just adapt the test annotations according to your preferred test framework. After each test, we'll take a screenshot and close the browser context to clean up resources. ## Parallel Testing with Java One of the great advantages of the TestingBot service is that you can run multiple tests simultaneously. This drastically shortens the total duration of your test suite, as multiple tests will run concurrently. Here's how you can set up parallel test execution using JUnit 5: ```java package com.example; import com.google.gson.JsonObject; import com.microsoft.playwright.*; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.parallel.Execution; import org.junit.jupiter.api.parallel.ExecutionMode; import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertTrue; // Enable parallel execution for the class @Execution(ExecutionMode.CONCURRENT) class ParallelPlaywrightTests { // Helper method to set up the browser private Browser setupBrowser(String browserName) { // Create capabilities JsonObject capabilities = new JsonObject(); JsonObject tbOptions = new JsonObject(); tbOptions.addProperty("key", "YOUR_KEY"); tbOptions.addProperty("secret", "YOUR_SECRET"); capabilities.add("tb:options", tbOptions); capabilities.addProperty("browserName", browserName); capabilities.addProperty("browserVersion", "latest"); // Create the playwright instance Playwright playwright = Playwright.create(); String wsEndpoint = "wss://cloud.testingbot.com/playwright?capabilities=" + java.net.URLEncoder.encode(capabilities.toString(), java.nio.charset.StandardCharsets.UTF_8); // Connect based on browser type BrowserType browserType; if (browserName.equals("chrome")) { browserType = playwright.chromium(); } else if (browserName.equals("firefox")) { browserType = playwright.firefox(); } else if (browserName.equals("webkit")) { browserType = playwright.webkit(); } else { throw new IllegalArgumentException("Unsupported browser: " + browserName); } return browserType.connect(new BrowserType.ConnectOptions() .setWsEndpoint(wsEndpoint)); } @Test void testHomepageInChrome() { // Each test manages its own browser instance try (Browser browser = setupBrowser("chrome")) { BrowserContext context = browser.newContext(); Page page = context.newPage(); try { page.navigate("https://testingbot.com/"); String title = page.title(); assertEquals("TestingBot: Cross Browser Testing and Mobile App Testing", title); page.screenshot(new Page.ScreenshotOptions() .setPath(java.nio.file.Paths.get("chrome-homepage.png"))); } finally { context.close(); } } } @Test void testPricingPageInFirefox() { // Each test manages its own browser instance try (Browser browser = setupBrowser("firefox")) { BrowserContext context = browser.newContext(); Page page = context.newPage(); try { page.navigate("https://testingbot.com/pricing"); String title = page.title(); assertEquals("Cross Browser Testing - Prices and plans available for test online.", title); page.screenshot(new Page.ScreenshotOptions() .setPath(java.nio.file.Paths.get("firefox-pricing.png"))); } finally { context.close(); } } } } ``` To enable parallel execution with JUnit 5, you need to configure it in your `junit-platform.properties` file: ```properties junit.jupiter.execution.parallel.enabled=true junit.jupiter.execution.parallel.mode.default=concurrent junit.jupiter.execution.parallel.mode.classes.default=concurrent ``` With TestNG, you can configure parallel execution in your test XML file: ```xml ``` The examples above demonstrate how to run multiple tests concurrently against different browsers on the TestingBot infrastructure. Each test manages its own browser instance, allowing multiple tests to run in parallel. Depending on how many tests you run in parallel, you can drastically shorten your total test duration and get faster feedback from your tests. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/playwright/dotnet # Playwright & .NET Playwright for .NET provides a powerful API for browser automation that can be used for end-to-end testing with C# and other .NET languages. See the example below on how to use .NET with Playwright to run tests on the TestingBot browser grid. More information about Playwright for .NET can be found on the [Playwright .NET documentation](https://playwright.dev/dotnet/docs/intro). ## Installation To get started with Playwright for .NET, you'll need to install the relevant NuGet packages for your preferred test framework: [MSTest](https://testingbot.com#) [NUnit](https://testingbot.com#) [xUnit](https://testingbot.com#) To install the Playwright NuGet packages for MSTest, run the following commands in your terminal: ```bash dotnet new mstest dotnet add package Microsoft.Playwright dotnet add package Microsoft.Playwright.MSTest dotnet build pwsh bin/Debug/net8.0/playwright.ps1 install ``` On macOS/Linux, use: ```bash playwright install ``` To install the Playwright NuGet packages for NUnit, run the following commands in your terminal: ```bash dotnet new nunit dotnet add package Microsoft.Playwright dotnet add package Microsoft.Playwright.NUnit dotnet build pwsh bin/Debug/net8.0/playwright.ps1 install ``` On macOS/Linux, use: ```bash playwright install ``` To install the Playwright NuGet packages for xUnit, run the following commands in your terminal: ```bash dotnet new xunit dotnet add package Microsoft.Playwright dotnet build pwsh bin/Debug/net8.0/playwright.ps1 install ``` On macOS/Linux, use: ```bash playwright install ``` Note that there is no specific Microsoft.Playwright.xUnit package, but you can still use Playwright with xUnit. ### Code Generation Playwright provides a powerful code generation tool that helps you quickly create test scripts by recording your interactions with the browser. To use it, run the following command: ```bash pwsh bin/Debug/net8.0/playwright.ps1 codegen https://testingbot.com ``` On macOS/Linux, use: ```bash playwright codegen https://testingbot.com ``` This will open a browser window where you can interact with the webpage, and Playwright will generate C# code based on your actions. You can then copy this code and use it as a starting point for your tests. ### Running Tests To run your tests, use the standard `dotnet test` command: ```bash dotnet test ``` You can also run specific test classes or methods: ```bash dotnet test --filter "FullyQualifiedName=PlaywrightTests.PlaywrightTestingBotTest" ``` [NUnit](https://testingbot.com#) [MSTest](https://testingbot.com#) [xUnit](https://testingbot.com#) Example using NUnit: ```csharp using System; using System.Text.Json.Nodes; using System.Threading.Tasks; using System.Web; using Microsoft.Playwright; using Microsoft.Playwright.NUnit; using NUnit.Framework; namespace PlaywrightTests { [Parallelizable(ParallelScope.Self)] public class PlaywrightTestingBotTest : PageTest { private IBrowser _browser; private IBrowserContext _context; private IPage _page; [SetUp] public async Task SetupTestAsync() { // Create the capabilities object with authentication information var capabilities = new JsonObject { ["browserName"] = "chrome", ["browserVersion"] = "latest", ["platform"] = "WIN10", ["tb:options"] = new JsonObject { ["key"] = "YOUR_KEY", ["secret"] = "YOUR_SECRET" } }; // Create the playwright instance var playwright = await Playwright.CreateAsync(); // Connect to TestingBot var wsEndpoint = $"wss://cloud.testingbot.com/playwright?capabilities={HttpUtility.UrlEncode(capabilities.ToJsonString())}"; _browser = await playwright.Chromium.ConnectAsync(new BrowserTypeConnectOptions { WsEndpoint = wsEndpoint }); // Create a context and page _context = await _browser.NewContextAsync(); _page = await _context.NewPageAsync(); } [TearDown] public async Task TearDownTestAsync() { if (_page != null) { // Take a screenshot at the end of the test await _page.ScreenshotAsync(new PageScreenshotOptions { Path = $"screenshot-{Guid.NewGuid()}.png" }); await _context?.CloseAsync(); await _browser?.CloseAsync(); } } [Test] public async Task ShouldCheckPageTitle() { await _page.GotoAsync("https://testingbot.com/"); var title = await _page.TitleAsync(); Assert.AreEqual("TestingBot: Cross Browser Testing and Mobile App Testing", title); } } } ``` Example using MSTest: ```csharp using System; using System.Text.Json.Nodes; using System.Threading.Tasks; using System.Web; using Microsoft.Playwright; using Microsoft.VisualStudio.TestTools.UnitTesting; namespace PlaywrightTests { [TestClass] public class PlaywrightTestingBotTest { private static IPlaywright _playwright; private IBrowser _browser; private IBrowserContext _context; private IPage _page; [TestInitialize] public async Task TestInitialize() { // Create the capabilities object with authentication information var capabilities = new JsonObject { ["browserName"] = "chrome", ["browserVersion"] = "latest", ["platform"] = "WIN10", ["tb:options"] = new JsonObject { ["key"] = "YOUR_KEY", ["secret"] = "YOUR_SECRET" } }; // Create the playwright instance _playwright = await Playwright.CreateAsync(); // Connect to TestingBot var wsEndpoint = $"wss://cloud.testingbot.com/playwright?capabilities={HttpUtility.UrlEncode(capabilities.ToJsonString())}"; _browser = await _playwright.Chromium.ConnectAsync(new BrowserTypeConnectOptions { WsEndpoint = wsEndpoint }); // Create a context and page _context = await _browser.NewContextAsync(); _page = await _context.NewPageAsync(); } [TestCleanup] public async Task TestCleanup() { if (_page != null) { // Take a screenshot at the end of the test await _page.ScreenshotAsync(new PageScreenshotOptions { Path = $"screenshot-{Guid.NewGuid()}.png" }); await _context?.CloseAsync(); await _browser?.CloseAsync(); _playwright.Dispose(); } } [TestMethod] public async Task ShouldCheckPageTitle() { await _page.GotoAsync("https://testingbot.com/"); var title = await _page.TitleAsync(); Assert.AreEqual("TestingBot: Cross Browser Testing and Mobile App Testing", title); } } } ``` Example using xUnit: ```csharp using System; using System.Text.Json.Nodes; using System.Threading.Tasks; using System.Web; using Microsoft.Playwright; using Xunit; namespace PlaywrightTests { public class PlaywrightTestingBotTest : IAsyncLifetime { private IPlaywright _playwright; private IBrowser _browser; private IBrowserContext _context; private IPage _page; public async Task InitializeAsync() { // Create the capabilities object with authentication information var capabilities = new JsonObject { ["browserName"] = "chrome", ["browserVersion"] = "latest", ["platform"] = "WIN10", ["tb:options"] = new JsonObject { ["key"] = "YOUR_KEY", ["secret"] = "YOUR_SECRET" } }; // Create the playwright instance _playwright = await Playwright.CreateAsync(); // Connect to TestingBot var wsEndpoint = $"wss://cloud.testingbot.com/playwright?capabilities={HttpUtility.UrlEncode(capabilities.ToJsonString())}"; _browser = await _playwright.Chromium.ConnectAsync(new BrowserTypeConnectOptions { WsEndpoint = wsEndpoint }); // Create a context and page _context = await _browser.NewContextAsync(); _page = await _context.NewPageAsync(); } public async Task DisposeAsync() { if (_page != null) { // Take a screenshot at the end of the test await _page.ScreenshotAsync(new PageScreenshotOptions { Path = $"screenshot-{Guid.NewGuid()}.png" }); await _context?.CloseAsync(); await _browser?.CloseAsync(); _playwright.Dispose(); } } [Fact] public async Task ShouldCheckPageTitle() { await _page.GotoAsync("https://testingbot.com/"); var title = await _page.TitleAsync(); Assert.Equal("TestingBot: Cross Browser Testing and Mobile App Testing", title); } } } ``` This example will use Playwright to connect to a Chrome browser in the TestingBot cloud. It will open the TestingBot website, retrieve the title of the page and verify if the title is correct. You can also use test frameworks like MSTest or xUnit with Playwright, following a similar approach. Just adapt the test attributes according to your preferred test framework. After each test, we'll take a screenshot and close the browser context to clean up resources. ## Parallel Testing with .NET One of the great advantages of the TestingBot service is that you can run multiple tests simultaneously. This drastically shortens the total duration of your test suite, as multiple tests will run concurrently. Here's how you can set up parallel test execution using NUnit: ```csharp using System; using System.Text.Json.Nodes; using System.Threading.Tasks; using System.Web; using Microsoft.Playwright; using NUnit.Framework; namespace PlaywrightTests { [TestFixture] [Parallelizable(ParallelScope.All)] // Enable parallel execution public class ParallelPlaywrightTests { // Helper method to set up the browser private async Task SetupBrowserAsync(string browserName) { // Create capabilities var capabilities = new JsonObject { ["browserName"] = browserName, ["browserVersion"] = "latest", ["tb:options"] = new JsonObject { ["key"] = "YOUR_KEY", ["secret"] = "YOUR_SECRET" } }; // Create the playwright instance var playwright = await Playwright.CreateAsync(); // Determine which browser type to use IBrowserType browserType; switch (browserName.ToLower()) { case "chrome": browserType = playwright.Chromium; break; case "firefox": browserType = playwright.Firefox; break; case "webkit": case "safari": browserType = playwright.Webkit; break; default: throw new ArgumentException($"Unsupported browser: {browserName}"); } // Connect to TestingBot var wsEndpoint = $"wss://cloud.testingbot.com/playwright?capabilities={HttpUtility.UrlEncode(capabilities.ToJsonString())}"; return await browserType.ConnectAsync(new BrowserTypeConnectOptions { WsEndpoint = wsEndpoint }); } [Test] public async Task TestHomepageInChrome() { // Each test manages its own browser instance using var playwright = await Playwright.CreateAsync(); var browser = await SetupBrowserAsync("chrome"); try { var context = await browser.NewContextAsync(); var page = await context.NewPageAsync(); try { await page.GotoAsync("https://testingbot.com/"); var title = await page.TitleAsync(); Assert.AreEqual("TestingBot: Cross Browser Testing and Mobile App Testing", title); await page.ScreenshotAsync(new PageScreenshotOptions { Path = "chrome-homepage.png" }); } finally { await context.CloseAsync(); } } finally { await browser.CloseAsync(); } } [Test] public async Task TestPricingPageInFirefox() { // Each test manages its own browser instance using var playwright = await Playwright.CreateAsync(); var browser = await SetupBrowserAsync("firefox"); try { var context = await browser.NewContextAsync(); var page = await context.NewPageAsync(); try { await page.GotoAsync("https://testingbot.com/pricing"); var title = await page.TitleAsync(); Assert.AreEqual("Cross Browser Testing - Prices and plans available for test online.", title); await page.ScreenshotAsync(new PageScreenshotOptions { Path = "firefox-pricing.png" }); } finally { await context.CloseAsync(); } } finally { await browser.CloseAsync(); } } } } ``` To enable parallel execution with NUnit, add the following to your `NUnit.runsettings` file: ```xml 4 ``` For MSTest, you can configure parallel execution in your `.runsettings` file: ```xml 4 ClassLevel ``` The examples above demonstrate how to run multiple tests concurrently against different browsers on the TestingBot infrastructure. Each test manages its own browser instance, allowing multiple tests to run in parallel. Depending on how many tests you run in parallel, you can drastically shorten your total test duration and get faster feedback from your tests. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/playwright/ruby # Playwright & Ruby The [playwright-ruby-client](https://github.com/YusukeIwaki/playwright-ruby-client) gem provides a Ruby interface for Playwright, enabling browser automation for testing and web scraping. See the example below on how to use Ruby with Playwright to run tests on the TestingBot browser grid. More information about Playwright for Ruby can be found on the [playwright-ruby-client documentation](https://playwright-ruby-client.vercel.app/docs/article/getting_started). ## Installation To get started, add the playwright-ruby-client gem to your Gemfile: ```ruby gem 'playwright-ruby-client' ``` Then run: ```bash bundle install ``` Or install it directly: ```bash gem install playwright-ruby-client ``` ## Running a Test Below is an example of how to connect to the TestingBot Playwright grid using the playwright-ruby-client gem: [Chrome](https://testingbot.com#) [Firefox](https://testingbot.com#) [WebKit](https://testingbot.com#) ```ruby require 'playwright' require 'json' require 'uri' capabilities = { 'tb:options' => { 'key' => ENV['TB_KEY'], 'secret' => ENV['TB_SECRET'] }, 'browserName' => 'chrome', 'browserVersion' => 'latest', 'platform' => 'WIN10' } ws_endpoint = "wss://cloud.testingbot.com/playwright?capabilities=#{URI.encode_www_form_component(capabilities.to_json)}" Playwright.connect_to_browser_server(ws_endpoint) do |browser| page = browser.new_page page.goto('https://testingbot.com/') puts page.title page.screenshot(path: 'screenshot.png') browser.close end ``` ```ruby require 'playwright' require 'json' require 'uri' capabilities = { 'tb:options' => { 'key' => ENV['TB_KEY'], 'secret' => ENV['TB_SECRET'] }, 'browserName' => 'firefox', 'browserVersion' => 'latest', 'platform' => 'WIN10' } ws_endpoint = "wss://cloud.testingbot.com/playwright?capabilities=#{URI.encode_www_form_component(capabilities.to_json)}" Playwright.connect_to_browser_server(ws_endpoint) do |browser| page = browser.new_page page.goto('https://testingbot.com/') puts page.title page.screenshot(path: 'screenshot.png') browser.close end ``` ```ruby require 'playwright' require 'json' require 'uri' capabilities = { 'tb:options' => { 'key' => ENV['TB_KEY'], 'secret' => ENV['TB_SECRET'] }, 'browserName' => 'safari', 'browserVersion' => 'latest', 'platform' => 'VENTURA' } ws_endpoint = "wss://cloud.testingbot.com/playwright?capabilities=#{URI.encode_www_form_component(capabilities.to_json)}" Playwright.connect_to_browser_server(ws_endpoint) do |browser| page = browser.new_page page.goto('https://testingbot.com/') puts page.title page.screenshot(path: 'screenshot.png') browser.close end ``` This example will use Playwright to connect to a browser in the TestingBot cloud. It will open the TestingBot website, print the title of the page and save a PNG screenshot. ## Using with RSpec You can integrate Playwright with RSpec for running your tests. Here's an example: ```ruby require 'playwright' require 'json' require 'uri' RSpec.describe 'TestingBot Playwright Test' do let(:capabilities) do { 'tb:options' => { 'key' => ENV['TB_KEY'], 'secret' => ENV['TB_SECRET'] }, 'browserName' => 'chrome', 'browserVersion' => 'latest', 'platform' => 'WIN10' } end let(:ws_endpoint) do "wss://cloud.testingbot.com/playwright?capabilities=#{URI.encode_www_form_component(capabilities.to_json)}" end it 'verifies the page title' do Playwright.connect_to_browser_server(ws_endpoint) do |browser| page = browser.new_page page.goto('https://testingbot.com/') expect(page.title).to include('TestingBot') browser.close end end end ``` ## Parallel Testing with Ruby One of the great advantages of the TestingBot service is that you can run multiple tests simultaneously. This drastically shortens the total duration of your test suite, as multiple tests will run concurrently. You can use the `parallel_tests` gem to run your RSpec tests in parallel: ```bash gem install parallel_tests ``` Then run your tests with: ```bash parallel_rspec spec/ ``` Here's an example of running tests across multiple browsers in parallel using threads: ```ruby require 'playwright' require 'json' require 'uri' browsers = [ { name: 'chrome', type: :chromium, platform: 'WIN10' }, { name: 'firefox', type: :firefox, platform: 'WIN10' }, { name: 'safari', type: :webkit, platform: 'VENTURA' } ] threads = browsers.map do |browser_config| Thread.new do capabilities = { 'tb:options' => { 'key' => ENV['TB_KEY'], 'secret' => ENV['TB_SECRET'] }, 'browserName' => browser_config[:name], 'browserVersion' => 'latest', 'platform' => browser_config[:platform] } ws_endpoint = "wss://cloud.testingbot.com/playwright?capabilities=#{URI.encode_www_form_component(capabilities.to_json)}" Playwright.connect_to_browser_server(ws_endpoint) do |browser| page = browser.new_page page.goto('https://testingbot.com/') puts "#{browser_config[:name]}: #{page.title}" page.screenshot(path: "screenshot-#{browser_config[:name]}.png") browser.close end end end threads.each(&:join) ``` This example runs tests on Chrome, Firefox, and Safari simultaneously, taking screenshots on each browser. ## Mark Test Status You can mark your test as passed or failed in the TestingBot dashboard by using the `testingbot_executor` command: ```ruby require 'playwright' require 'json' require 'uri' capabilities = { 'tb:options' => { 'key' => ENV['TB_KEY'], 'secret' => ENV['TB_SECRET'] }, 'browserName' => 'chrome', 'browserVersion' => 'latest', 'platform' => 'WIN10' } ws_endpoint = "wss://cloud.testingbot.com/playwright?capabilities=#{URI.encode_www_form_component(capabilities.to_json)}" Playwright.connect_to_browser_server(ws_endpoint) do |browser| page = browser.new_page begin page.goto('https://testingbot.com/') if page.title.include?('TestingBot') # Mark test as passed page.evaluate("testingbot_executor: #{JSON.generate({ action: 'setSessionStatus', arguments: { passed: true, reason: 'Title matched' } })}") else raise 'Title did not match' end rescue => e # Mark test as failed page.evaluate("testingbot_executor: #{JSON.generate({ action: 'setSessionStatus', arguments: { passed: false, reason: e.message } })}") ensure browser.close end end ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/playwright/options # Playwright Options This page lists various options that you can use to customise your Playwright sessions on TestingBot. The [Playwright Capabilities generator](https://testingbot.com/support/web-automate/playwright/capabilities) allows you to easily generate the necessary capabilities for your tests. ## Basic Settings ### Browser Name The name of the browser to run your Playwright test on. TestingBot currently supports these browserNames: - **Chrome** - **Edge** - **Firefox** - **Safari (WebKit)** [Chrome](https://testingbot.com#) [Edge](https://testingbot.com#) [Firefox](https://testingbot.com#) [WebKit](https://testingbot.com#) ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET }, browserName: 'chrome', browserVersion: 'latest' } const browser = await playwright.chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET }, browserName: 'edge', browserVersion: 'latest' } const browser = await playwright.chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET }, browserName: 'firefox', browserVersion: 'latest' } const browser = await playwright.firefox.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET }, browserName: 'safari', browserVersion: 'latest' } const browser = await playwright.webkit.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` ### Browser Version The version of the browser to run your Playwright test on. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. Other Options: - `"version": "*"` : If you use `*` as version, TestingBot will pick a random version. - `"version": "latest"` : TestingBot will automatically take the latest version. You can also use `latest-1`, `latest-2`, ... to test on the next most recent versions. For example, if the current latest Chrome version is 89 and you use `latest-2`, then the test will run on Chrome 87. - `"version": "<=16"` : TestingBot will pick a version smaller than or equal to the version you specify with `<=`. `"version": ">=16"` : TestingBot will pick a version higher than or equal to the version you specify with `>=`. ### Browser Platform The OS/platform for the test VM which will run the Playwright test. ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET }, browserName: 'chrome', platform: 'WIN10' } const browser = await pw.chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` Possible values: - `WIN11` - `WIN10` - `LINUX` - `TAHOE` - `SEQUOIA` - `VENTURA` - `MONTEREY` - `BIGSUR` ### Make a video of your tests By default we record a video of your test, which is accessible in the member area. If you do not wish to have this, you can disable it with this option. Video will not slow down your test considerably. ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, screenrecorder: false }, browserName: 'chrome', browserVersion: 'latest' } await pw.chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` Value Type | Default Value || boolean | `true` | ### Test Privacy Make the test results for this test public so that everyone can access the results. ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, public: true }, browserName: 'chrome', browserVersion: 'latest' } await pw.chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` Value Type | Default Value || boolean | `false` | ### Blacklist hostnames The hostnames you specify will be pointed to localhost instead of their real destination. This means you can speed up tests by blocking third party content which you don't need and slows down your test. ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, blacklist: 'site1.com,site2.com' }, browserName: 'chrome', browserVersion: 'latest' } await pw.chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` Value Type || string (comma-separated) | ### Customize Logging By default, TestingBot records logs of all Playwright actions. Set this option to `false` if you don't want TestingBot to record anything (for example, if you have sensitive data). ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, recordLogs: true }, browserName: 'chrome', browserVersion: 'latest' } await pw.chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` Value Type | Default Value | Possible Values: || boolean | `true` | `true`, `false` | ### Custom Time Zones Change the Time Zone of the Virtual Machine to the Time Zone you specify. You can find a [list of timezones on Wikipedia](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones). Only location names are supported (not their paths). See some examples below: ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, timeZone: 'Etc/UTC' }, browserName: 'chrome', browserVersion: 'latest' } await pw.chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` Value Type | Default Value | Possible Values: || string | "Etc/UTC" | [List of timezones](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) | ### Change Screen Resolution Will adjust the screen resolution before your test. ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, 'screen-resolution': '1280x1024' }, browserName: 'chrome', browserVersion: 'latest' } await pw.chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` Value Type | Default Value | || string | "1280x1024" | Platform \| Resolutions \|\| Windows/Linux \| - 800x600 - 1024x768 - 1152x864 - 1280x768 - 1280x800 - 1280x960 - 1280x1024 - 1400x1050 - 1600x1200 - 1680x1050 - 1920x1080 - 1920x1200 - 2560x1440 \| \| macOS \| - 800x600 - 1024x768 - 1280x768 - 1280x800 - 1280x960 - 1280x1024 - 1366x768 - 1440x900 - 1600x900 - 1600x1200 - 1680x1050 - 1920x1080 - 1920x1200 - 2048x1536 \| | ### Geolocation Testing We provide an option where you can specify from which country you'd like to run the test from. Once you specify this option, the virtual machine we provision for your test will be configured to use a proxy in the country you specified. ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, geoCountryCode: 'US' }, browserName: 'chrome', browserVersion: 'latest' } await pw.chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` We currently support the following country codes: - **'\*'** : this will take a random country from the list below - **'AU'** : Australia - **'BE'** : Belgium - **'BR'** : Brazil - **'CA'** : Canada - **'CL'** : Chile - **'FR'** : France - **'DE'** : Germany - **'IN'** : India - **'IT'** : Italy - **'JP'** : Japan - **'NO'** : Norway - **'SG'** : Singapore - **'ZA'** : South Africa - **'SE'** : Sweden - **'CH'** : Switzerland - **'AE'** : United Arab Emirates - **'GB'** : United Kingdom - **'US'** : United States ### Upload This option allows you to upload a file to the test VM running the Playwright test. TestingBot will first download the file from the URL you specified and save it on the test VM so your test can use the file. ```javascript const pw = require('playwright-core') const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, upload: 'https://testingbot.com/assets/logo-head.png', uploadFilepath: '/tmp/test.log' }, browserName: 'chrome', browserVersion: 'latest' } ;(async () => { const browser = await pw.chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) const page = await browser.newPage() await page.goto('http://localhost:3000') await page.locator('input[name="file-upload"]').click() await page.locator('input[name="file-upload"]').setInputFiles('/tmp/test.png') await page.click('upload-button') await browser.close() })() ``` You need to specify both the download URL as `upload` and use `uploadFilepath` to indicate where TestingBot needs to put the file. We recommend using these directories to save your files: For Windows (`C:\test\`), for Linux/macOS: (`/tmp/`). ### Change Playwright Version By default, TestingBot will use the same Playwright version as the one that is running your test. If for some reason you need to specify a different version, you can use the `playwrightVersion` option. ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, name: 'First Test' }, 'playwright:options': { playwrightVersion: '1.54.1' }, browserName: 'chrome', browserVersion: 'latest' } await pw.chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` ### Change Test Name Add a name to this test, which will show up in our member area and API. ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, name: 'First Test' }, browserName: 'chrome', browserVersion: 'latest' } await pw.chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` Value Type | Default Value || string | unnamed test | ### Group Tests A key you can use to group certain tests in the same build (for example in Jenkins). The builds will appear in our member area. ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, build: 'First Build' }, browserName: 'chrome', browserVersion: 'latest' } await pw.chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` Value Type || string | ### Timeout Control the timeout setting of the session. This setting, specified in seconds, controls when a session times out. A timeout occurs when a session has not received any commands from your Playwright test for the xx amount of seconds specified. If a certain Playwright command takes longer than the specified setting, the session will automatically close. ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, timeout: 90 }, browserName: 'chrome', browserVersion: 'latest' } await pw.chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` Value Type | Default Value || int (specify number of seconds) | 90 seconds | ### Maximum Test Duration The maximum duration for a single test. This is a safeguard to prevent bad tests from using up your credits. We generally recommend to keep tests short (less than 10 minutes). It's better to split up large tests in smaller individual tests. This keeps your tests fast and allows for more parallelization of your tests. ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, maxduration: 1800 }, browserName: 'chrome', browserVersion: 'latest' } await pw.chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` Value Type | Default Value || int (specify number of seconds) | 1800 seconds (30 minutes) | ### Custom Metadata Send along custom data, for example your release, server, commit hash, ... This will show up on the test detail page in the TestingBot member area. ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, extra: 'Extra Information' }, browserName: 'chrome', browserVersion: 'latest' } await pw.chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` Value Type || string | ### Startup Flags You can customize the startup flags of the browser by passing in extra flags. This is identical to passing `args` with Playwright. ```javascript const pw = require('playwright-core') const startupFlags = [] startupFlags.push('--window-size=1280,720') startupFlags.push('--hide-scrollbars') ;(async () => { const browser = await pw.chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com` + `/playwright?key=api_key&secret=api_secret&browserName=chrome&startupFlags=${JSON.stringify(startupFlags)}` }) const page = await browser.newPage() await page.goto('https://testingbot.com') await browser.close() })() ``` ### TestingBot Tunnel You can specify the `tunnelIdentifier` to [use Playwright with TestingBot Tunnel](https://testingbot.com/support/web-automate/playwright#tunnel). ```javascript const pw = require('playwright-core') ;(async () => { const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET, tunnelIdentifier: 'myPlaywrightTunnel' }, browserName: 'chrome', browserVersion: 'latest' } const browser = await pw.chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) const context = await browser.newContext() const page = await context.newPage() await page.goto('http://localhost:3000/') await page.screenshot({ path: 'screenshot.png' }) await browser.close() })() ``` By default we support proxying these ports for localhost testing: `[443, 80, 8080, 3030, 3000, 3001, 3400]`. If you need additional ports, please pass an array of port numbers to either `localHttpPorts` or `localHttpsPorts` ### Firefox Preferences You can pass in the [firefoxPreferences](https://playwright.dev/docs/api/class-browsertype#browser-type-launch-option-firefox-user-prefs) which Playwright accepts. ```javascript const pw = require('playwright-core') ;(async () => { const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET }, 'playwright:options': { firefoxPreferences: '' }, browserName: 'firefox', browserVersion: 'latest' } await pw.firefox.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) })() ``` ### Devtools You can pass in the [devTools](https://playwright.dev/docs/api/class-browsertype#browser-type-launch-option-devtools) which Playwright accepts. This is a Chromium-only option which decides whether to auto-open a Developer Tools panel for each tab during the test. ```javascript const pw = require('playwright-core') ;(async () => { const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET }, 'playwright:options': { devtools: true }, browserName: 'chrome', browserVersion: 'latest' } await pw.chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) })() ``` ### Patchright Patchright is a modified version of Playwright that tries to avoid detection by anti-bot systems. It patches several leaks, which are used by anti-scraper systems to detect Playwright automation. You can use Patchright with TestingBot, by passing in the `patchRightVersion` option, specifying the version of Patchright you want to use. ```javascript const pw = require('playwright-core') ;(async () => { const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET }, 'playwright:options': { patchRightVersion: '1.52.5' }, browserName: 'chrome', browserVersion: 'latest' } await pw.chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) })() ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/playwright/recorder # Playwright Recorder Playwright offers a feature where you can record the actions in your browser to a Playwright script file. This allows you to easily create tests by recording your own actions, similar to [Selenium IDE](https://testingbot.com/support/web-automate/codeless-automation/add-test). To get started, install these packages: ```bash npm install playwright ``` ## Example Now you can record your actions to a Playwright file: ```bash npx playwright codegen --target javascript -o example.js https://testingbot.com ``` This command will start a Chromium browser and will record every action you take, in JavaScript format, to a file called `example.js`. You can choose to save the file in a different format, these are the options: - javascript - python - python-async - csharp The `example.js` file can contain something similar to this: ```javascript const { chromium } = require('playwright'); (async () => { const browser = await chromium.launch({ headless: false }); const context = await browser.newContext(); // Open new page const page = await context.newPage(); // Go to https://testingbot.com/ await page.goto('https://testingbot.com/'); // Click text=Take your automated and manual testing to the next level. await page.click('text=Take your automated and manual testing to the next level.'); // Click img[alt="Automated Testing"] await page.click('img[alt="Automated Testing"]'); // Close page await page.close(); // --------------------- await context.close(); await browser.close(); })(); ``` ## Run on TestingBot To run the generated file on a browser in the TestingBot grid, you'll have to modify the script. ### Before ```javascript const browser = await chromium.launch(); ``` ### After ```javascript const capabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET }, browserName: 'chrome', browserVersion: 'latest' } const browser = await chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` }) ``` See our [Playwright Codegen](https://testingbot.com/resources/articles/playwright-recorder) article for a walkthrough of recording from Visual Studio Code, including how to add assertions to a recorded test. ## Playwright Agents An alternative approach to recording tests would be to generate tests from your existing code. Playwright comes with [3 agents](https://playwright.dev/docs/test-agents) that can be used to create tests. - **planner** explores the app and produces a Markdown test plan - **generator** transforms the Markdown plan into the Playwright Test files - **healer** executes the test suite and automatically repairs failing tests, for example due to bad/missing locators To get started, use this command to initialize the agent definitions: ```bash npx playwright init-agents --loop=claude ``` You can now use your AI tool of choice to generate tests and point these to use a remote TestingBot browser by replacing the `launch` command with a `connect` command, as shown in the previous sections. For example with a prompt: "Create one or more Playwright tests from the markdown test plan that was generated." ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/playwright/codeceptjs # CodeceptJS & Playwright CodeceptJS is an end-to-end testing framework which provides support for many automation drivers, including Playwright. To get started, please install these packages: ```bash npm install codeceptjs playwright --save ``` To create a sample project from scratch, you can use this command: ```bash npx codeceptjs init ``` ## Configure CodeceptJS with Playwright To configure CodeceptJS, please edit `codecept.conf.js` and make sure to add the `Playwright` helper. ```javascript { // .. helpers: { Playwright: { url: "http://localhost", browser: 'chromium', chromium: { browserWSEndpoint: "wss://cloud.testingbot.com?key=YOUR_KEY&secret=YOUR_SECRET", cdpConnection: true } } } // .. } ``` You can use the `browserWSEndpoint` URL to instruct CodeceptJS to use TestingBot's Playwright grid to run the tests on. ## Specifying browser and version To specify on which browser, browser version and platform your CodeceptJS + Playwright test should run, you can include both a `browserName` and `browserVersion` in the `browserWSEndpoint` URL. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Run your first CodeceptJS + Playwright test The example test below will open the TestingBot website and check to see if the word 'TestingBot' is included on the page. ```javascript Feature('Test'); Scenario('test testingbot homepage', ({ I }) => { I.amOnPage('https://testingbot.com'); I.see('TestingBot'); }); ``` Now you can run this first test on TestingBot. ```bash npx codeceptjs run --steps ``` ``` Test -- test something I am on page "https://testingbot.com" I see "TestingBot" ✔ OK in 1017ms OK | 1 passed // 10s ``` This test will appear in your [Member Dashboard](https://testingbot.com/members). Test details will include a video of the test and any logs that were generated. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/playwright/playwright-parallel-testing # Parallel Testing **Parallel testing** lets you execute multiple Playwright tests at the same time, dramatically reducing your overall test suite duration and accelerating feedback cycles. On TestingBot's browser grid, you can easily run Playwright tests concurrently across more than 100 browser and device combinations. This enables you to validate your web application in multiple environments, all in parallel. The guide below will walk you through setting up and running Playwright tests concurrently using TestingBot’s scalable infrastructure. ## Parallel Example In the example below, we'll start various Playwright sessions on different browser versions, in the TestingBot cloud. ```javascript const pw = require('playwright-core'); const doConnect = async (cap) => { cap['tb:options'] = { key: process.env.TB_KEY, secret: process.env.TB_SECRET } const browser = await pw.chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(cap))}`, }); const context = await browser.newContext(); const page = await context.newPage(); await page.goto('https://testingbot.com/'); await page.screenshot({ path: `screenshot-${cap.browserName}-${cap.browserVersion}.png` }); await browser.close(); } const capabilities = [ { 'browserName': 'chrome', 'browserVersion': 'latest', 'platform': 'WIN10', }, { 'browserName': 'chrome', 'browserVersion': 'latest-1', 'platform': 'LINUX', }, { 'browserName': 'edge', 'browserVersion': 'latest', 'platform': 'SEQUOIA', }] ;(async () => { await Promise.all( capabilities.map((cap) => doConnect(cap)) ) console.log('All tests finished!'); })(); ``` You can use `latest` and `latest-x` to automatically run on the latest browser version. ### Queueing Depending on your account subscription, you can run multiple tests in parallel. If you exceed your maximum allocated slots, the connection will be queued until a slot frees up. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/playwright/browsers # Browser and OS versions available for Playwright testing The TestingBot browser grid consists of +100 browser and OS combinations compatible with Playwright testing. We currently support these browser vendors with Playwright testing: - Chrome - Microsoft Edge - Firefox - Safari (WebKit) ## Specifying browser combination To indicate on which browser you want your Playwright test to run, please use the following parameters as a `browserWSEndpoint` value. - `browserName` - `browserVersion` - `platform` Capability | Description | Possible Values: || `browserName` | The name of the browser | `chrome`, `firefox`, `edge` and `webkit` (for Safari) | | `browserVersion` | The version of the browser | - **Chrome:** 72 and above - **Edge:** 80 and above - **Firefox:** 110 and above - **Safari:** will use the latest WebKit version bundled with Playwright You can use `latest`, `latest-1`, `latest-{x}` to test on the latest versions of the browser. | | `platform` | The Operating System where the test will run | - **Windows:** `WIN11`, `WIN10` - **macOS:** `CATALINA`, `BIGSUR`, `VENTURA`, `SONOMA`, `SEQUOIA`, `TAHOE` - `LINUX` | There was no release for version `82` of Chrome and Edge, so please do not specify this version. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/playwright/change-screen-resolution # Change desktop screen resolution with Playwright The TestingBot Desktop VMs (Windows, macOS and Linux) all support changing the screen-resolution before a test starts. We currently support the following screen resolutions: Value Type | Default Value | || string | "1280x1024" | Platform \| Resolutions \|\| Windows/Linux \| - 800x600 - 1024x768 - 1152x864 - 1280x768 - 1280x800 - 1280x960 - 1280x1024 - 1400x1050 - 1600x1200 - 1680x1050 - 1920x1080 - 1920x1200 - 2560x1440 \| \| macOS \| - 800x600 - 1024x768 - 1280x768 - 1280x800 - 1280x960 - 1280x1024 - 1366x768 - 1440x900 - 1600x900 - 1600x1200 - 1680x1050 - 1920x1080 - 1920x1200 - 2048x1536 \| | To change the screen resolution, please specify the `screen-resolution` parameter in the `browserWSEndpoint` URL. ```javascript const pw = require('playwright-core'); (async () => { const browser = await pw.chromium.connect({ wsEndpoint: 'wss://cloud.testingbot.com/playwright?key=api_key&secret=api_secret&browserName=chrome&browserVersion=latest&screen-resolution=1600x900', }); const context = await browser.newContext(); const page = await context.newPage(); await page.goto('https://testingbot.com/'); await page.screenshot({ path: 'screenshot.png' }); await browser.close(); })(); ``` Changing the screen resolution does not resize the viewport. To change the viewport with Playwright, please use `page.setViewportSize`. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/playwright/change-browser-options # Browser Options This document will guide you in specifying custom browser options, which can be used with Playwright testing. You can change the browser start arguments for a Chromium-based browser, or pass user-preferences for the Firefox browser. ## Set args for Chromium browsers In the example below we'll set custom browser `args` which will be used by the browser upon startup. ```javascript const pw = require('playwright-core') const startupFlags = [] startupFlags.push('--window-size=1280,720') startupFlags.push('--hide-scrollbars') ;(async () => { const browser = await pw.chromium.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright` + `?key=api_key&secret=api_secret&browserName=chrome&startupFlags=${JSON.stringify(startupFlags)}` }) const page = await browser.newPage() await page.goto('https://testingbot.com') await browser.close() })() ``` You can use any of the [Chromium commandline switches](https://peter.sh/experiments/chromium-command-line-switches/). ## Set Firefox user preferences If you are running a Playwright test on Firefox, you can set the Firefox User Preferences. ```javascript const pw = require('playwright-core') const firefoxPrefs = { 'browser.autoFocus': true } ;(async () => { const browser = await pw.firefox.connect({ wsEndpoint: `wss://cloud.testingbot.com/playwright` + `?key=api_key&secret=api_secret&browserName=firefox&firefoxUserPrefs=${JSON.stringify(firefoxPrefs)}` }) const page = await browser.newPage() await page.goto('https://testingbot.com') await browser.close() })() ``` For more information, please see [Firefox Preferences](https://support.mozilla.org/en-US/kb/about-config-editor-firefox). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/playwright/get-session-details # Get Playwright Session Details When a Playwright test runs on TestingBot, it will generate various artifacts such as logs, video, screenshots and other files. TestingBot will save these files for later retrieval, through our member dashboard or our [REST API](https://testingbot.com/support/api). To fetch these details through our REST API, you will need to know the unique identifier for each test. TestingBot provides a `testingbot_executor` which you can use to retrieve the `sessionId`, to use in your REST API calls. ## Fetch the sessionId To fetch the sessionId in your Playwright script, you can fetch the `sessionId` from the `getSessionDetails` method. See the example below, where we run a Playwright script, fetch the `getSessionDetails` at the end and update the test status via the TestingBot REST API. ```javascript const TestingBot = require('testingbot-api'); const tb = new TestingBot({ api_key: 'api_key', api_secret: 'api_secret' }) const playwright = require('playwright-core'); (async () => { const browser = await playwright.chromium.connect({ wsEndpoint: 'wss://cloud.testingbot.com/playwright?key=api_key&secret=api_secret&browserName=chrome&browserVersion=latest&platform=WIN10' }) const context = await browser.newContext() const page = await context.newPage() await page.goto('https://testingbot.com') const testingBotResponse = await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'getSessionDetails'})}`) const sessionId = testingBotResponse.sessionId const testData = { "test[success]" : "1", "test[status_message]" : "test" }; await new Promise((resolve) => tb.updateTest(testData, sessionId, function(error, testDetails) { resolve(testDetails) })) await browser.close() })() ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/playwright/test-results # Test Results After running a Playwright test on TestingBot, you can see the test result in the [TestingBot Dashboard](https://testingbot.com#dashboard) overview. You can click each test to see a [detailed overview](https://testingbot.com#single) for each specific test. By default, a Playwright test runs in headless mode, so no video will be available. You can change this by [enabling video recording](https://testingbot.com/support/web-automate/playwright/options#screenrecorder). ## The Playwright Dashboard The dashboard contains an overview of all your previous test runs, together with some meta-data: - **Test Name** : you can pass this name via our [REST API](https://testingbot.com/support/web-automate/playwright#update). - **Pass/Failure state** : you can [indicate](https://testingbot.com/support/web-automate/playwright#update) whether this test passed or failed. - **Browser/Device** : the browser or device that was used by the test. TestingBot provides [more than 100 different browser/OS combinations](https://testingbot.com/support/web-automate/playwright/browsers). - **Start Date** : when we received the call to start your test. - **Test Duration** : how long it took for your test to run. This is the actual run time, potential queueing time is not included. ## Single Playwright Test Detail Page For each test, we have a test detail page, which shows an overview of all the data we have: - Test Name - [session details](https://testingbot.com/support/web-automate/playwright/get-session-details) of the test - the pass/failure state, if you used our [REST API](https://testingbot.com/support/web-automate/playwright#update) - the browser/device that was used by the test - the date when the test started - how long the test took to run - logs for the test - a video recording of the test (if the test ran with [recording enabled](https://testingbot.com/support/web-automate/playwright/options#screenrecorder)) - the user who started the test in your team ## Viewing results without the dashboard You can [share, embed and link to builds or individual tests](https://testingbot.com/support/other/sharing) without having to visit the dashboard. This is useful when you want to create your own reporting. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/playwright/debug-tests # Debug Failing Playwright Tests This document will guide you in using various techniques to debug Playwright tests running on TestingBot. We provide several features which can help you in locating problems or debugging Playwright tests. ## Interactive Session Our interactive session feature allows you to pause a running Playwright script, and inspect the remote browser with the included Developer Tools. Simply click the **Pause Test** button on the test page while the test is running and you will be navigated to a remote browser. You can now use your mouse and keyboard to interact with the browser and debug potential issues. One important thing to remember is that you will need to add a sleep/wait in your code, so that your Playwright commands stop being sent when debugging. ## Video Recording Every Playwright test that runs on TestingBot comes with a [recorded video](https://testingbot.com/support/web-automate/playwright/options#screenrecorder) of the entire screen, providing you run the Playwright test in `headful` mode. The video is available in our member area, where you can click each Playwright test. Or you can [fetch the video](https://testingbot.com/support/web-automate/playwright/get-session-details), together with other details, through our REST API. ## Log Files For every Playwright test running on TestingBot, we collect the browser's log. This log can be useful for pinpointing certain issues with your Playwright script. Log files are available in our member area, where you can click each Playwright test. Or you can [fetch the logs](https://testingbot.com/support/web-automate/playwright/get-session-details), together with other details, through our REST API. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/playwright/mark-test-status # Mark test status with Playwright The Playwright tests you run on TestingBot are using the `WebSocket` protocol to send commands to a remote browser. These commands instruct the remote browser what to do: click, type, resize, ... Because the logic of your test cases is inside your test script, TestingBot has no way of knowing if a test failed or passed. There is a way to send back the test status to TestingBot, by including a code snippet at the end of your test, where you know whether the test passed or failed. Below we'll go over two methods to send back the test status to TestingBot, so that we can show the pass/failure state in the dashboard and in the [TestingBot analytics](https://testingbot.com/support/analytics/guide). ## Mark test status from within a test script TestingBot has created a custom Playwright command which you can use in your Playwright scripts. You can pass in a `boolean` to indicate whether the test **passed** or **failed**. ```javascript await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'setSessionStatus',arguments: { passed: false, reason: 'Title did not match' }})}`) ``` This command will set the test as failed, with a specific reason which will be visible on the TestingBot test detail page. You can pass in the reason for a failure with the `reason` field. This command will return a JSON object with the `sessionId` of the test, which can be used with our [REST-API](https://testingbot.com/support/api). You would typically call this at the end of each test, where you know if the test failed or passed. For example, see the code snippet below where we use Chai's `expect` to verify if the test passed or not. ```javascript const pw = require('playwright-core') const expect = require('chai').expect (async () => { const browser = await pw.chromium.connect({ wsEndpoint: 'wss://cloud.testingbot.com/playwright?key=api_key&secret=api_secret&browserName=chrome&browserVersion=latest', }) const context = await browser.newContext() const page = await context.newPage() await page.goto('https://testingbot.com/') const title = await page.title() try { expect(title).to.equal("Cross Browser Testing and Mobile App Testing. Test on web browsers, mobile emulators, simulators and physical mobile devices.", 'Expected page title is incorrect!') // This is ok, so mark the test as passed await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'setSessionStatus',arguments: {passed: true, reason: 'Title matched'}})}`) } catch { // Test failed, mark the test as failed on TestingBot await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'setSessionStatus',arguments: {passed: false, reason: 'Title did not match'}})}`) } await browser.close() })() ``` ## Mark test status with API You can fetch the `sessionId` during an active Playwright test. This unique id can be used during or after the test to set the pass/fail state for the test. Marking the test as passed or failed will show the success state in the TestingBot test dashboard, on the individual test detail page and in the [TestingBot analytics](https://testingbot.com/support/analytics/guide) pages. To fetch the `sessionId`, please use the sample code below. ```javascript const testingBotResponse = await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'getSessionDetails'})}`) ``` The `testingBotResponse` variable will contain a `sessionId` property which you can use: ```json { "sessionId": "......" } ``` With the `sessionId`, you can now mark the test as passed or failed via the [TestingBot REST-API](https://testingbot.com/support/api): ```bash $ curl "https://api.testingbot.com/v1/tests/:sessionId" \ -X PUT \ -d "test[success]=1" \ -u key:secret ``` Learn more about this API call and other API calls on the TestingBot [API documentation](https://testingbot.com/support/api). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/playwright/browser-extension-testing # Testing Browser Extensions with Playwright Testing browser extensions on Chrome, Edge and Firefox is possible with the combination of Playwright and TestingBot. Please see the information below on how to load the extension before your test starts and how to perform UI testing with Playwright against these extensions. Currently these types of extensions are supported: Browser | Supported Format(s) | Example || [Chrome](https://testingbot.com#chrome) | - `.zip` file, containing the extension source code (`manifest.json`, `src/ directory`, ...) - `.crx` file | - `https://.../extension.zip` - `tb://sample-extension` | | [Firefox](https://testingbot.com#firefox) | - `.zip` file, containing the extension source code (`manifest.json`, `src/ directory`, `lib/ directory`, ...) - `.xpi` file | - `https://.../extension.zip` - `tb://sample-extension` | | [Edge](https://testingbot.com#edge) | - `.zip` file, containing the extension source code (`manifest.json`, `src/ directory`, ...) - `.crx` file | - `https://.../extension.zip` - `tb://sample-extension` | ## TestingBot Storage With [TestingBot Storage](https://testingbot.com/support/api/storage#upload), you can upload your extensions to the TestingBot servers. The advantage of this is that our test VMs can immediately download your extension from the TestingBot network, which is much faster than downloading from the public internet. The API call will return a unique identifier, similar to `tb://efe932jfk`. You can then use this identifier with the capability: `"load-extension" : "tb://..."` ## Chrome Browser Extension Testing with Playwright You can test your `.crx` extension by [uploading the extension](https://testingbot.com#storage) and using the URL of the extension with a `load-extension` capability. Because of Playwright limitations, we will need to use CDP to connect to Chrome based browsers in order to load extensions. ```javascript const pw = require('playwright-core') ;(async () => { const browser = await pw.chromium.connectOverCDP('wss://cloud.testingbot.com/?key=api_key&secret=api_secret&browserName=chrome&browserVersion=latest&load-extension=tb://path-to-extension') const defaultContext = browser.contexts()[0] const page = defaultContext.pages()[0] await page.goto('https://www.google.com') // test if your extension does something await browser.close() })(); ``` ### UI Testing Chrome Extensions You can test the UI and functionality of your extension in a remote Chrome browser. ```javascript const pw = require('playwright-core') ;(async () => { const tbCapabilities = { browserName: 'chrome', platform: 'LINUX', key: process.env.TB_KEY, secret: process.env.TB_SECRET, 'load-extension': 'tb://path-to-extension' } const wsEndpoint = `wss://cloud.testingbot.com/?capabilities=${encodeURIComponent( JSON.stringify(tbCapabilities) )}`; const browser = await pw.chromium.connectOverCDP(wsEndpoint) const defaultContext = browser.contexts()[0] const page = defaultContext.pages()[0] await page.goto('chrome-extension://lliapakmekmldfaphofjcnbccemlgeoa/src/pages/popup/index.html') await browser.close() })(); ``` ## Firefox Browser Extension Testing with Playwright To test your `.xpi` extension, you can specify a public URL or a [TestingBot Storage](https://testingbot.com#storage) URL in the `load-extension` capability. ```javascript const pw = require('playwright-core') ;(async () => { const browser = await pw.firefox.connect({ wsEndpoint: 'wss://cloud.testingbot.com/playwright?key=api_key&secret=api_secret&browserName=firefox&browserVersion=latest&load-extension=tb://path-to-extension' }) const context = await browser.newContext() const page = await context.newPage() await page.goto('https://www.google.com') // test if your extension does something await browser.close() })(); ``` ### UI Testing Firefox Extensions You can test the UI and functionality of your extension in a remote Mozilla Firefox browser. If you want to test the UI of your Firefox extension, the `moz-extension://` URL, you will need to specify the extension ID in the URL. Because TestingBot installs the extension as a temporary extension, the id will always be different as it is generated as a random uuid(4). TestingBot provides a capability to hardcode your Firefox extension to a fixed uuid which you've generated, by using the `firefoxExtensionMap` capability. To use this, you could set the ID in the `manifest.json` of the extension: ```javascript const pw = require('playwright-core') ;(async () => { const tbCapabilities = { browserName: 'firefox', platform: 'LINUX', key: process.env.TB_KEY, secret: process.env.TB_SECRET, 'load-extension': 'tb://path-to-extension', 'firefoxExtensionMap': { 'addon@testingbot.com': '3d9b1639-77fb-44a1-888a-6d97d773e96b' } } const wsEndpoint = `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent( JSON.stringify(tbCapabilities) )}`; const browser = await pw.firefox.connect({ wsEndpoint }) const context = await browser.newContext() const page = await context.newPage() await page.goto('moz-extension://3d9b1639-77fb-44a1-888a-6d97d773e96b/src/pages/popup/index.html') await browser.close() })(); ``` ## Edge Browser Extension Testing with Playwright You can test your Microsoft Edge extension by [uploading the extension](https://testingbot.com#storage) and using the URL of the extension with a `load-extension` capability. Because of Playwright limitations, we will need to use CDP to connect to Edge based browsers in order to load extensions. ```javascript const pw = require('playwright-core') ;(async () => { const browser = await pw.chromium.connectOverCDP('wss://cloud.testingbot.com/?key=api_key&secret=api_secret&browserName=microsoftedge&browserVersion=latest&load-extension=tb://path-to-extension') const defaultContext = browser.contexts()[0] const page = defaultContext.pages()[0] await page.goto('https://www.google.com') // test if your extension does something await browser.close() })(); ``` ### UI Testing Edge Extensions You can test the UI and functionality of your extension in a remote Edge browser. ```javascript const pw = require('playwright-core') ;(async () => { const tbCapabilities = { browserName: 'edge', platform: 'WIN10', key: process.env.TB_KEY, secret: process.env.TB_SECRET, 'load-extension': 'tb://path-to-extension' } const wsEndpoint = `wss://cloud.testingbot.com/?capabilities=${encodeURIComponent( JSON.stringify(tbCapabilities) )}`; const browser = await pw.chromium.connectOverCDP(wsEndpoint) const defaultContext = browser.contexts()[0] const page = defaultContext.pages()[0] await page.goto('extension://lliapakmekmldfaphofjcnbccemlgeoa/src/pages/popup/index.html') await browser.close() })(); ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/playwright/mobile # Mobile Playwright Testing Playwright offers [experimental support for Android automation](https://playwright.dev/docs/api/class-android). TestingBot provides a grid of remote Android devices and emulators. You can use Playwright scripts to automate Chrome for Android browsers and Android WebViews. ```bash npm i playwright-core ``` ```javascript const { _android } = require('playwright-core'); (async () => { const tbCapabilities = { browserName: 'chrome', platformName: 'Android', deviceName: 'Pixel 8', browserVersion: '14.0', key: process.env.TB_KEY, secret: process.env.TB_SECRET } const wsEndpoint = `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(tbCapabilities))}` const device = await _android.connect(wsEndpoint) const context = await device.launchBrowser() const [page] = context.pages() await page.goto('https://testingbot.com') const resp = await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'setSessionStatus',arguments: {passed: true }})}`); await page.close() await context.close() await device.close() })(); ``` The example above will connect to a remote Pixel 8 device running Android 14. It will open the Chrome browser and navigate to TestingBot's homepage. After running this test, a video together with logs will be available in the TestingBot member area. ## Specify Android Device You can specify a specific Android device with the `deviceName` and `version` capabilities. These capabilities can target either Android emulators, or physical Android devices. ![Android OS selected](https://testingbot.com/assets/environments/svg/android-1b3d235e17522e2238e1c55b4e9cdd41219f653cd958ea065877d3813f32ffc3.svg) Android 16.0 › Pixel 9 Loading environments... Please wait while we load the available browsers and platforms. We offer special parameters which you can use to allocate a device: Regex Input | Result || `"Pixel.*"` | This will allocate any available Pixel device (phone) | | `"*"` | This will allocate a random available Android device | | `"Galaxy [20-21]"` | This will allocate either a Samsung Galay 20 or 21 | Some Examples: ```javascript // find any Samsung Galaxy, except 23 const tbCapabilities = { browserName: 'chrome', platformName: 'Android', deviceName: '^(Galaxy.*)(?!23)$', } // find any device which name starts with Pixel const tbCapabilities = { browserName: 'chrome', platformName: 'Android', deviceName: 'Pixel.*', } ``` ## Testing WebViews Playwright can connect and automate Android WebViews. Please see the example below, where we use `device.webView({ pkg: 'org.chromium.webview_shell' })`. ```javascript const { _android } = require('playwright-core'); (async () => { const tbCapabilities = { browserName: 'chrome', platformName: 'Android', deviceName: 'Pixel 8', browserVersion: '14.0', key: process.env.TB_KEY, secret: process.env.TB_SECRET } const wsEndpoint = `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(tbCapabilities))}` const device = await _android.connect(wsEndpoint) const context = await device.launchBrowser() // Launch an application with WebView. await device.shell('am force-stop org.chromium.webview_shell'); await device.shell('am start org.chromium.webview_shell/.WebViewBrowserActivity'); // Get the WebView. const webview = await device.webView({ pkg: 'org.chromium.webview_shell' }); // Fill the URL box. await device.fill({ res: 'org.chromium.webview_shell:id/url_field', }, 'testingbot.com'); await device.press({ res: 'org.chromium.webview_shell:id/url_field', }, 'Enter'); const page = await webview.page(); await page.waitForNavigation({ url: /.*testingbot.*/ }); console.log(await page.title()); await page.close() await context.close() await device.close() })(); ``` ## Playwright Emulation Playwright allows you to emulate specific devices, such as mobile phones or tablets. This is similar in functionality to other Devtools, where the viewport size is changed, together with the user-agent and other settings. You can specify your own custom device configurations, where you can [override the viewport size](https://playwright.dev/docs/emulation#viewport). Create a file called `playwright.config.js` with the following contents: ```javascript import { defineConfig, devices } from '@playwright/test'; export default defineConfig({ projects: [ { name: 'chromium', use: { ...devices['Desktop Chrome'], viewport: { width: 1280, height: 720 } } }, { name: 'Mobile Safari', use: { ...devices['iPhone 15'] } } ] }); ``` You will also need to create a `testingbot.config.js` file, which will contain code to connect to the TestingBot Playwright server: ```javascript export function getConnectWsEndpoint(userCapabilities) { const defaultCapabilities = { 'tb:options': { key: process.env.TB_KEY, secret: process.env.TB_SECRET } } const capabilities = { ...defaultCapabilities, ...userCapabilities } const connectUrl = `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` return connectUrl } ``` You can now include a [sample Playwright test](https://testingbot.com/support/web-automate/playwright/playwright-test#example) and run it on the TestingBot cloud. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/cypress # Cypress Testing Cypress provides easy and fast testing on Chrome, Firefox and Microsoft Edge. TestingBot has a CLI tool that will run your Cypress tests on the TestingBot browser grid. Perform Cypress automation in the cloud, with TestingBot's browser grid. This will allow you to run Cypress tests in parallel on a wide collection of browsers; test on different versions and operating systems. By default the CLI is using **Cypress v13.3.3** ([change](https://testingbot.com/support/web-automate/cypress/version)). Testing is available on these platforms: key | value | possible values || platform | The Operating System (OS) you want to run the tests on. | - `WIN11` - `WIN10` - `TAHOE` - `SEQUOIA` - `SONOMA` - `VENTURA` - `MONTEREY` - `BIGSUR` - `CATALINA` - `MOJAVE` | | browserName | The browser you want to run the tests on. | `chrome`, `firefox` and `microsoftedge` | | version | The version of the browser | **Chrome:** version `70` to `latest` **Firefox:** version `70` to `latest` **Edge:** version `80` to `latest` **(WIN10 only)** | | screenResolution | Specify a screen resolution for your test. | See a [list of screen resolutions](https://testingbot.com/support/web-automate/selenium/test-options#screenresolution) | ## Install the CLI To get started with the CLI, please install it with `npm` or `yarn`: ```console $ npm install testingbot-cypress-cli ``` You can find the repository for the CLI on [github@testingbot-cypress-cli](https://github.com/testingbot/testingbot-cypress-cli) ## Configuration Once installed, you'll need to provide the CLI with a `testingbot.json` configuration file. This file contains various settings, including the path to your Cypress tests. To generate a configuration file, you can use the `init` command: ```console $ testingbot-cypress init ``` Please add the `key` and `secret` you obtained from the TestingBot [member dashboard](https://testingbot.com/members/user/edit) in the `testingbot.json` file. Next, please supply `cypress_proj_dir` with the path to the folder that contains the `cypress.json` or `cypress.config.js` file. As an example, you can use the [Cypress Kitchen Sink Example](https://github.com/cypress-io/cypress-example-kitchensink). Before you can run your tests, you need to specify on which browser (version) and platform you'd like to test. You can specify the browsers you want to use in the `testingbot.json` file: ```json { "auth": { "key": "TESTINGBOT_KEY", "secret": "TESTINGBOT_SECRET" }, "browsers": [ { "browserName": "chrome", "platform": "VENTURA", "version": "latest" }, { "browserName": "chrome", "platform": "WIN10", "version": "latest" } ], "run_settings": { "cypress_project_dir": ".../cypress/cypress-example-kitchensink", "build_name": "cypress-build", "npm_dependencies": {}, "package_config_options": {}, "start_tunnel": true, "local_ports": [8080], "realTimeLogs": true, "parallel": 5 }, "tunnel_settings": { "verbose": false } } ``` ## Run tests You are now ready to run your Cypress tests on TestingBot. Instead of using the Cypress test runner, you can use TestingBot's test runner. Run the following command: ```console $ testingbot-cypress run ``` This will zip your Cypress tests, upload them to TestingBot and instruct TestingBot to start running the tests on the platform(s) you specified in the configuration file. The CLI will then display a link to view a live feed of your Cypress tests and will return realtime logs of the Cypress client running on TestingBot. Once the tests have finished running, the CLI will exit with `code 1` (failure) or `code 0` (success). ## Example As an example, you can use [cypress-example-kitchensink](https://github.com/cypress-io/cypress-example-kitchensink) These are the steps you need to follow to run the example on TestingBot: - Clone the kitchensink repo and install the dependencies: `npm install` - Start the kitchensink example: `npm start` - Add the local port used by kitchensink (8080) to the `local_ports` array, this way the TestingBot VMs can access the port. - Change the `cypress_project_dir` to point to the path where you installed the example. - Run the tests: ```console $ testingbot-cypress run ``` - Test results will appear in your console and on the [TestingBot dashboard](https://testingbot.com/members). ## Options Several options are available in the `testingbot.json` file. - `cypress_project_dir`: this needs to be specified and is the directory (absolute path) where your cypress configuration file is located. `cypress.json` or `cypress.config.js`. - `build_name`: this will set the build name on TestingBot. All Cypress tests will be grouped under this name. - `npm_dependencies`: you can list the NPM dependencies that your Cypress project requires here: ```json "npm_dependencies": { "npm-package-1": "^1.0.1", "npm-package-2": "^1.2.3-beta", } ``` - `package_config_options`: the config options for each of the npm dependencies you specified: ```json "package_config_options": { "npm-package-1": { "option1": true, "option2": "some-value" } } ``` - `start_tunnel`: should the CLI start a [TestingBot Tunnel](https://testingbot.com/support/tunnel) to access websites on your local computer/network? - `local_ports`: an array of ports that Cypress is using on your local computer. Use this in combination with `start_tunnel = true` to run tests from your local computer or network. - `realTimeLogs`: should the CLI display a real-time feed of the Cypress logs? - `tunnel_settings`: add additional [Tunnel options](https://testingbot.com/support/tunnel/commandline) to the TestingBot tunnel process. ## Environment Variables The CLI supports various Environment Variables: - You can use `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` to pass the TestingBot credentials to the CLI. - If you set `TESTINGBOT_CI=1` then the CLI will output the `TestingBotSessionID` which can be used in combination with one of our [CI plugins](https://testingbot.com/integrations#ci). ## Limitations While TestingBot supports most of the features Cypress provides, there are some that are not yet supported. Below is a list of features we currently do not support. If you'd like us to add support for these, please [reach out](https://testingbot.com/contact/new). - Cypress Component Testing - Module API - Test Replay ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/cypress/authentication # Authentication Options To use the TestingBot CLI, you'll need to authenticate with our Service. You can authenticate in two ways: - Specify [environment variables](https://testingbot.com#environment) - [Adding the credentials](https://testingbot.com#file) in the `testingbot.json` file ## Environment Variables You can set the following two environment variables before using the CLI: ```bash TESTINGBOT_KEY=your_testingbot_key ``` ```bash TESTINGBOT_SECRET=your_testingbot_secret ``` The CLI will automatically pick up these two variables and use them to authenticate. You can find these two values in our [member dashboard](https://testingbot.com/members/user/edit). ## Credentials in Config File You can set the [TestingBot Key and Secret values](https://testingbot.com/members/user/edit) in the `testingbot.json` file: ```json { "auth": { "key": "TESTINGBOT_KEY", "secret": "TESTINGBOT_SECRET" }, ... } ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/cypress/reporting # Reporting The TestingBot Cypress CLI contains an option to generate reports from your Cypress Runs, similar to other reporters such as mocha-awesome. To generate a report, please specify the `--reporter` flag: Argument | Shorthand Argument | Accepted Values || `--reporter` | `-r` | You can choose to use the `json` reporter, or the `junit` reporter | ## JUnit Reporter The JUnit reporter is especially useful if you intend to run your Cypress tests in a CI such as Jenkins. Here's an example of how to generate a JUnit report: ```console testingbot-cypress run --reporter junit ``` Most CI systems will be able to parse the report. The CI can then show a list of specs/tests that you ran on TestingBot, together with information such as: - How long each spec took to run - Whether the spec passed or not - The full error message if a spec failed By default, the reporter will output the report in a file called `test-results.xml`, in the same directory where you run the CLI command from. You can choose to change the location of this file with: ```console testingbot-cypress run --reporter junit --reporter-options mochaFile=/home/user/reports/my-report.xml ``` ## JSON Reporter The JSON reporter will output all data to a JSON file, similar to a report generated with [mochawesome-report-generator](https://github.com/adamgruber/mochawesome-report-generator) Here's an example of how to generate a JSON report: ```console testingbot-cypress run --reporter json ``` By default, the reporter will output the report in a file called `results.json`, in the same directory where you run the CLI command from. You can choose to change the location of this file with: ```console testingbot-cypress run --reporter json --reporter-options mochaFile=/home/user/reports/my-report.json ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/cypress/variables # Cypress Test Environment Variables You can easily setup test environment variables to be used with Cypress and TestingBot Cypress CLI. There are currently three ways to do this: 1. Using the `env` option key in your Cypress configuration file 2. Create a `cypress.env.json` file and specify your environment variables 3. Use the `--env` CLI flag with `testingbot-cypress` ## Cypress Configuration File Cypress allows you to specify your environment variables in the Cypress config file (`cypress.json` or `cypress.config.js`), under the `env` option key. You can learn more about this option in the [Cypress Documentation](https://docs.cypress.io/guides/guides/environment-variables.html#Option-1-configuration-file). ```json "env": { "signup_url": "/signup", "pricing_url": "/pricing" } ``` You can then access these values in Cypress with: ```javascript Cypress.env() // {signup_url: '/signup', pricing_url: '/pricing'} Cypress.env('signup_url') // '/signup' Cypress.env('pricing_url') // '/pricing' ``` ## cypress.env.json You can also create your own `cypress.env.json` file. TestingBot's Cypress CLI will send this together with your specs to instruct Cypress to use these environment variables. Values in this file will overwrite conflicting environment variables in your Cypress configuration file. See the [Cypress env.json documentation](https://docs.cypress.io/guides/guides/environment-variables.html#Option-2-cypress-env-json). ```json { "host": "testingbot.com", "api": "api.testingbot.com" } ``` You can then access these values in Cypress with: ```javascript Cypress.env() // {host: 'testingbot.com', api: 'api.testingbot.com'} Cypress.env('host') // 'testingbot.com' Cypress.env('api') // 'api.testingbot.com' ``` ## CLI Flag You can also pass a comma-separated list of environment variables with the TestingBot Cypress CLI. We will pass these values to the [Cypress Runner](https://docs.cypress.io/guides/guides/environment-variables.html#Option-4-env). Argument | Shorthand Argument | Possible values || `--env` | `-e` | comma-separated environment variables | ```bash testingbot-cypress run --env host=testingbot.com,api=api.testingbot.com ``` You can then access these values in Cypress with: ```javascript Cypress.env() // {host: 'testingbot.com', api: 'api.testingbot.com'} Cypress.env('host') // 'testingbot.com' Cypress.env('api') // 'api.testingbot.com' ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/cypress/capabilities # Browsers & Platforms TestingBot allows you to run your Cypress tests on hundreds of browser & platform combinations. You can use the `browsers` field in the `testingbot.json` file to indicate on which browsers & platforms you want to run your Cypress tests: key | value | possible values || platform | The Operating System (OS) you want to run the tests on. | `WIN11`, `WIN10`, `TAHOE`, `SEQUOIA`, `VENTURA` and `MONTEREY` | | browserName | The browser you want to run the tests on. | `chrome`, `firefox` and `microsoftedge` | | version | The version of the browser | **Chrome:** version `70` to `latest` **Firefox:** version `70` to `latest` **Edge:** version `80` to `latest` **(WIN10 only)** | ## Example Configuration Please see the example below on how to specify multiple browsers: ```json { ... "browsers": [ { "browserName": "chrome", "platform": "VENTURA", "version": "latest" }, { "browserName": "chrome", "platform": "WIN10", "version": "latest" }, { "browserName": "firefox", "platform": "MONTEREY", "version": "latest-1" } ], ... } ``` You can use `latest` or `latest-X` to always test on the latest versions, or specify a specific browser version. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/cypress/reference # CLI Reference Below you can find an overview of all the various options that the TestingBot Cypress CLI provides. Command | Purpose || `init` | This will initialize a new `testingbot.json` file, which contains the default configuration necessary to run tests | | `run` | [Start running your tests](https://testingbot.com#run) | ## Start running your tests To run your Cypress tests, please use this command, together with any of the optional arguments: ```bash testingbot-cypress run ``` Argument | Shorthand Argument | Accepted Values | Description || `--config-file` | `-cf` | The path to the `testingbot.json` config file. By default, this will look in the current directory for a `testingbot.json` file. | The `testingbot.json` config file is used to communicate with TestingBot. | | `--group` | | Accepts a `string`, can be any name. For example a build identifier from your CI. | Group all these tests in the TestingBot dashboard. Similar to our [build option](https://testingbot.com/support/web-automate/selenium/test-options#idletimeout). | | `--headless` | | Requires no arguments. If you pass this option, TestingBot will run the Cypress tests in headless mode. | With headless mode, tests will run without a UI and may run faster. | | `--parallel` | | Requires an integer, for example `--parallel 5` If you pass this option, TestingBot will split up your specs and run these on separate machines. | Depending on your plan, you can run your Cypress tests in parallel on TestingBot. This significantly cuts down the time it takes to run your full Cypress test suite. | | `--specs` | `-s` | Accepts a comma-separated list of files, or a glob. | Runs a subset of your specs. See [our Cypress Specs documentation](https://testingbot.com/support/web-automate/cypress/specs) for more information. | | `--env` | `-e` | Accepts a comma-separated list of environment variables | Sets the Cypress Environment Variables. See [our Cypress Environment Variables documentation](https://testingbot.com/support/web-automate/cypress/variables) for more information. | ## Stopping your tests While the TestingBot CLI is running your tests, you may decide that you want to stop the actively running tests. You can do this easily: - use `CTRL+C` in your terminal window - send a `SIGHUP` kill command to the CLI process ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/cypress/specs # Cypress Spec Files By default, Cypress will run all the spec files it can find. You can specify a `--specs` option with the TestingBot CLI to only run a subset of your specs on TestingBot. Argument | Shorthand Argument | Possible values || `--specs` | `-s` | comma-separated paths or globs for your spec files | ```bash testingbot-cypress run --specs "cypress/integration/examples/window.spec.js" testingbot-cypress run --specs "cypress/integration/register/**/*" testingbot-cypress run --specs "cypress/integration/examples/files.spec.js,cypress/integration/examples/navigation.spec.js" ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/cypress/upload # Upload Options The TestingBot Cypress CLI will automatically collect and upload your Cypress tests and configuration files, to run these on the TestingBot Browser Grid. These files are uploaded to [TestingBot Storage](https://testingbot.com/support/api/storage#upload), where they will only be used for running tests (in parallel) on TestingBot. However, you may want to exclude certain files from being uploaded, if: - The ZIP file is too large (over 150MB) - The upload is taking too long - You are zipping unnecessary files, for example `node_modules/*` If you're looking to only run a subset of tests, please see the [specs](https://testingbot.com/support/web-automate/cypress/specs) option. ## Exclude files with Run Settings To exclude certain files, please specify the file, directory, or a wildcard combination in the `run_settings` with `exclude`: ```json { ... "run_settings": { ... "exclude": ["some-folder/test.js", "static/*.pdf"] ... } ... } ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/cypress/tutorial # Cypress Tutorial This documentation page is a quick tutorial on how to set up an example app, with Cypress tests, and run Cypress Tests on multiple browsers in parallel on TestingBot. ## Install the Example The most popular example with Cypress is the [Cypress Kitchensink Example](https://github.com/cypress-io/cypress-example-kitchensink). You can [download](https://github.com/cypress-io/cypress-example-kitchensink/archive/master.zip) or clone it from GitHub: [Cypress v10 and higher](https://testingbot.com#) [Cypress v9 and lower](https://testingbot.com#) ```bash git clone https://github.com/cypress-io/cypress-example-kitchensink.git ``` ```bash git clone https://github.com/cypress-io/cypress-example-kitchensink.git git checkout ab10094ef7b199ae7febafec413a0626414bcd3c ``` Once you have the example on your computer, navigate to the directory and install its dependencies: ```bash cd cypress-example-kitchensink npm install ``` ## Configuration Once you have the example on your computer, you can install the TestingBot Cypress CLI: ```console npm install -g testingbot-cypress-cli ``` Next, you'll need to create a `testingbot.json` file which will hold the configuration and authentication details to connect with TestingBot. You can quickly generate a new configuration with the CLI: ```console testingbot-cypress init ``` This will create the json file in the current directory. Now you can edit the file: - **auth** : add your TestingBot `Key` and `Secret` (obtained from [member dashboard](https://testingbot.com/members/user/edit)) in the auth code block - **browsers** : specify the browsers that you want to run your test on - **run\_settings** : various options that you can configure to customize the Cypress Tests - **tunnel\_settings** : options to customize the [TestingBot tunnel](https://testingbot.com/support/tunnel), usually you will not need to modify this This is a good example configuration to continue with the tutorial: ```json { "auth": { "key": "TESTINGBOT_KEY", "secret": "TESTINGBOT_SECRET" }, "browsers": [ { "browserName": "chrome", "platform": "SEQUOIA", "version": "latest" } ], "run_settings": { "cypress_project_dir": ".../cypress/cypress-example-kitchensink", "build_name": "cypress-build", "npm_dependencies": {}, "package_config_options": {}, "start_tunnel": true, "local_ports": [8080], // optional: the port used by your app under test "realTimeLogs": true, "parallel": 2 }, "tunnel_settings": { "verbose": false } } ``` ## Setup the Example Next, you'll need to start up the kitchen-sink webserver. Simply go to the directory of the example, and run this command: ```console npm start ``` By default this app uses `localhost:8080` for its web server. That's the reason why we added `8080` to the `local_ports` configuration above. It allows the CLI to access this webserver. ## Run the tests You can now run the tests: ```console testingbot-cypress run ``` The test results, together with a video and logs of the tests, will be available in the [TestingBot dashboard](https://testingbot.com/members). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/cypress/version # Specifying Cypress Version TestingBot has installed several versions of Cypress on its VMs. You can request the TestingBot Cypress CLI to test on a specific version by specifying `cypressVersion` in the `testingbot.json` file. ```json { "run_settings": { "cypress_project_dir": "/path/to/directory-that-contains-cypress-config-file", "build_name": "build-name", "start_tunnel": true, "local_ports": [], "realTimeLogs": true, "cypressVersion": "13.3.3" } } ``` Currently we support these versions of Cypress: - 14.2.0 - 13.17.0 - 13.3.3 (default) - 12.5.1 - 10.0.0 - 9.1.0 - 8.0.0 If you'd like to test on another version, please [let us know](https://testingbot.com/contact/new). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/cypress/devtools # Run Cypress tests recorded with Chrome DevTools Chrome allows you to record your actions and generate scripts, all through Chrome DevTools. **Recorder** is a feature that can record (and replay) actions you make during a browser session. Various export formats are supported by **Recorder** , including the ability to export as a Cypress test script. ## Requirements Before you can get started, make sure to follow these requirements: - You need to be using Chrome 89 or higher. Previous versions do not come with the **Recorder** feature. - Install the Chrome Extension [Cypress Chrome Recorder](https://chrome.google.com/webstore/detail/cypress-chrome-recorder/fellcphjglholofndfmmjmheedhomgin/related) - TestingBot's Cypress CLI is installed, through [npm install testingbot-cypress-cli](https://testingbot.com/support/web-automate/cypress#install). ## Record your actions in Chrome ### Enable Recorder in Chrome First, we'll need to enable the recorder in Chrome. Please follow these steps: 1. Open Chrome and click the **More options** icon, then select **Developer Tools** from the More Tools option. 2. Once you're in the Developer Tools, click the **More options** (a vertical ellipses icon) and select **Recorder** from the **More Tools** option. 3. A Recorder tab will appear, with a button **Create a new recording**. You are now ready to start recording. ### Record Actions in Chrome We can now start recording a user flow in Chrome. Proceed with the following steps to start recording. 1. In **Recorder** , click the **Start new recording** button. 2. Enter a name in the **RECORDING NAME** input field. 3. Click the **Start a new recording** button at the bottom of the panel. 4. Perform various actions in Chrome while the recorder is running. 5. When you are finished, click the **End recording** button. ### Export recorded Actions Click the **Export** icon and select the **Export as a Cypress Test script** option. Save the exported file to your local disk, in a file called **cdp-testingbot.cy.js** in a folder **cypress/e2e/**. It should contain code similar to the code below. ```javascript describe("cdp-testingbot", () => { it("test with cdp on testingbot", () => { cy.viewport(1280, 824); cy.visit("https://testingbot.com/"); }); }); ``` Next, create a file **cypress.config.js** with the following content: ```javascript module.exports = { 'projectId': '4b7344', e2e: {}, } ``` ## Run the recorded Cypress actions on TestingBot To start running the test on TestingBot, you first need to create a `testingbot.json` file. ```console $ testingbot-cypress init ``` Please add the `key` and `secret` you obtained from the TestingBot [member dashboard](https://testingbot.com/members/user/edit) in the `testingbot.json` file. Before you can run your tests, you need to specify on which browser (version) and platform you'd like to test. You can specify the browsers you want to use in the `testingbot.json` file: ```json { "auth": { "key": "TESTINGBOT_KEY", "secret": "TESTINGBOT_SECRET" }, "browsers": [ { "browserName": "chrome", "platform": "WIN10", "version": "latest" } ], "run_settings": { "cypress_project_dir": "folder-here", "build_name": "cypress-recorded-actions", "npm_dependencies": {}, "package_config_options": {}, "realTimeLogs": true } } ``` Make sure to change the **cypress\_project\_dir** to the folder that contains your **cypress.config.js** file. Next, you can start running the tests. ```bash $ testingbot-cypress run ``` Test results will appear in your console and on the [TestingBot dashboard](https://testingbot.com/members). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/cypress/geolocation # Cypress Geolocation Testing TestingBot provides Cypress testing in the cloud. Together with TestingBot's IP GeoLocation feature, you are able to run Cypress tests from various countries around the world. The advantages of using geolocation testing with Cypress are that you can verify various scenarios in your tests, such as: - Test translated webpages - Verify localized pricing: different currencies - Test speed from various locations around the world We currently support the following geolocations for Cypress testing: - **'\*'** : this will take a random country from the list below - **'AU'** : Australia - **'BE'** : Belgium - **'BR'** : Brazil - **'CA'** : Canada - **'CL'** : Chile - **'FR'** : France - **'DE'** : Germany - **'IN'** : India - **'IT'** : Italy - **'JP'** : Japan - **'NO'** : Norway - **'SG'** : Singapore - **'ZA'** : South Africa - **'SE'** : Sweden - **'CH'** : Switzerland - **'AE'** : United Arab Emirates - **'GB'** : United Kingdom - **'US'** : United States ## Code Example You can specify the ISO code of the geolocation in the `testingbot.json` file, as a capability: ```json { ... "browsers": [ { "browserName": "chrome", "platform": "WIN10", "version": "latest", "testingbot.geoCountryCode": "DE" } ], ... } ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/cypress/typescript # Cypress Typescript Testing TestingBot offers a service similar to Cypress Cloud, where you can run Cypress tests on multiple browsers in the cloud, simultaneously. On this page, we will focus on how to run a Cypress test, built with Typescript, on TestingBot's remote browser grid. ## Setup Typescript and Cypress TestingBot provides a ready-to-use Github example: [typescript-cypress-testingbot](https://github.com/testingbot/typescript-cypress-testingbot). To get started, create a new project and add the necessary dependencies: - `npm init` - `npm install testingbot-cypress-cli typescript cypress --save-dev` - Next, create a `tsconfig.json` file with the following contents: - Let's also create a `cypress.config.ts` file (notice the `.ts` indicating this is a Typescript project). ## Typescript Example Now that everything is set up, we can add a first test. Create a `cypress` directory, which will contain 2 more directories: - `e2e` - `support` ### e2e.cy.ts file The `e2e.cy.ts` file in the `cypress/e2e` directory contains our test logic: ```javascript describe('TestingBot Demo test', () => { beforeEach(() => { cy.visit('https://testingbot.com') }) it('Verify title', () => { cy.title().should('include', 'TestingBot') }) }) ``` ### e2e.ts file Another file, `e2e.ts`, needs to be added in the `cypress/support` directory, containing: ```javascript // Import commands.js using ES2015 syntax: import './commands' ``` `commands.ts` can be an empty file for now. ## Run the Cypress test There's one more thing left for us to do. We need to create a `testingbot.json` file, which needs to contain a couple of things for TestingBot to run your test. To create the file, execute the following command in the base directory of your project: ```console testingbot-cypress init ``` You can now edit the `testingbot.json` file. ```json { "auth": { "key": "TESTINGBOT_KEY", "secret": "TESTINGBOT_SECRET" }, "browsers": [ { "browserName": "chrome", "platform": "VENTURA", "version": "latest" } ], "run_settings": { "cypress_project_dir": ".../cypress/cypress-example-kitchensink", "build_name": "cypress-build", "npm_dependencies": { "typescript": "^5.2.2" }, "package_config_options": {}, "start_tunnel": false, "realTimeLogs": true, "parallel": 1 }, "tunnel_settings": { "verbose": false } } ``` Make sure to specify the same typescript dependency in `npm_dependencies` as the one in your `package.json`. This will instruct TestingBot to use the same Typescript version for running your tests. Finally, to run the test, you can run the following command: ```console testingbot-cypress run ``` The test results, together with a video and logs of the tests, will be available in the [TestingBot dashboard](https://testingbot.com/members). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/web-automate/cypress/timezone # Configure Timezone During Cypress testing, you might want to test behaviour specific to a certain timezone. TestingBot allows you to configure the timezone of the machines that run your Cypress tests. Please use a timezone format from the [list of timezones](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones). ```json { ... "browsers": [ { "browserName": "chrome", "platform": "WIN10", "version": "latest", "timezone": "New_York" } ], ... } ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/visual-testing # Visual Testing with TestingBot Visual testing is a testing technique which aims to verify the visual aspects of a user interface (UI). If you want to make sure that no unintended visual mistakes occur, you will need to compare the visual appearance of your website or mobile app with a golden image (commonly known as baseline image). TestingBot makes it easy for you to run visual tests against the pages of your website, or the screens of your mobile app. You will be able to run visual tests across different browsers, OS versions and mobile devices. ## Getting started with UI Testing There are currently two options to use visual UI testing on TestingBot: 1. [Automated Visual Testing](https://testingbot.com/support/visual-testing/automated) Integrate TestingBot's UI comparison testing directly in your existing automated test scripts. 2. [Run scheduled visual tests](https://testingbot.com/support/visual-testing/codeless) Requires no code - simply specify URLs to your website and TestingBot will regularly inspect your website for UI changes. You'll receive an alert when a visual regression is detected. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/visual-testing/stabilize-screenshots # Stabilize Screenshots for Visual Testing During visual testing, we're comparing a screenshot with a previous screenshot. If a difference is detected, the visual test is considered a failed test. In order to avoid false positives during the comparison of two screenshots, it's important to make sure we prevent as much unwanted and unimportant dynamic data from the screenshot as possible. ## Freeze CSS Animations Right before TestingBot takes a screenshot, we will disable any CSS animations currently running on the page. TestingBot will freeze CSS `animation` and `transition` styles. Once the screenshot has been taken, the CSS animations will run just like before. ## Lazy Loaded Images Some images on the webpage might be marked as lazy loading images. If you are taking a full-page screenshot, TestingBot will force-load all images marked as lazy images, to make sure the entire page is visually tested. ## Mobile Statusbars When testing on (physical) mobile devices, a statusbar might be present on the screen, containing the current time. TestingBot will automatically remove the statusbar from the image, because otherwise the time indicator would trigger a visual difference compared to the baseline. ## Date/Time on webpages A webpage might display the current date or time. In order to avoid marking this as a difference, the date and time of the webpage is frozen to `2024-01-01T12:00:00Z`. Once the screenshot has been taken, the date and time is unfrozen. ## Native browser controls Right before TestingBot takes a screenshot, the caret and the scrollbar on the page will be hidden, in order to avoid false positives with focused text inputs. After the screenshot, these changes will be reverted. ## Visual Test Options TestingBot offers these features to stabilize your screenshots: ### Ignore Specific Regions You can ignore specific regions, defined in pixel coordinates, from the screenshot: ```json ignoreRegions = [{ "x1": number, "y1": number, "x2": number, "y2": number }] ``` ### Ignore Specific Selectors Pass an array of CSS Selectors to ignore these DOM elements during the visual check. ```json ignoreSelectors = ["body div.test", "#sidebar"] ``` ### Only screenshot a specific selector Pass a CSS selector to take a screenshot of this specific element only. ```json selector = "body div.test" ``` ## Scheduled Visual Tests For visual tests you schedule in the [dashboard](https://testingbot.com/members/uitest), two more settings decide whether a run fails. Both are per test and need no code. ### Mismatch threshold A run fails when more than this share of pixels differs from the baseline. New tests start at 1%; you can set anything between 0.1% and 5% on the test's settings page. The slider shows how many of your last ten runs would have passed at the value you pick, so you can size it against your own history rather than guessing. The comparison is tolerant of anti-aliasing before the threshold is applied: a pixel only counts as changed when a colour channel differs beyond a tolerance _and_ at least one neighbouring pixel changed too. Sub-pixel font jitter and isolated stray pixels do not fail a run. ### Ignored regions Parts of a page that change on their own, a carousel, a clock, a rotating ad, will differ on every run and fail a test for no useful reason. On the test's settings page you can drag a box over those areas of the baseline to ignore them. Ignored regions are painted out of both the screenshot and the baseline before they are compared, so nothing inside one can fail a run. You can copy the regions you drew onto every browser in the test, scaled to each baseline's width. After a failure, TestingBot also looks at the last three runs of that browser and proposes the regions that changed in every one of them. Nearby changes are merged into one rectangle, so a widget with several moving parts comes back as a single region rather than a box per line. These appear in amber on the settings page, drawn over the latest run, for you to accept or dismiss individually. Accepting one adds it to the ignored regions from the next run onwards. If most of the difference is the same on every run, the page itself has changed and you should approve a run as the new baseline instead. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/visual-testing/results # Results of a failing Visual Test When a visual test fails, you will see 3 images on the visual test result page: - An image of the current screenshot - An image of the baseline screenshot (the screenshot that you approved) - A diff image, showcasing the difference between the two images above ## Diff image A diff image will show the difference between the two images. See an example of such a result below. ![Visual Test Diff](https://testingbot.com/assets/support/visual/codeless/diff-1e13f4e968c249e56c3030418f0e61fce8fd68c76686fc7b2d7332732ad2bcb5.webp) ## Results of a successful Visual Test A successful visual test will show both the current screenshot and the baseline image. The two images are identical. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/visual-testing/codeless # Codeless Visual Testing TestingBot provides codeless visual testing, allowing you to perform scheduled visual checks against any website, without writing any code. Simply add a URL to the webpage you'd like to test, pick the browsers you want to test on and indicate how frequently the test should run. TestingBot will open the webpage on the browsers you indicated and perform a visual pixel-by-pixel check. When TestingBot detects a mismatch, you will be alerted. ## Setup Please follow these steps to create your first codeless visual test: 1. Click the **Visual Testing** menu item in the member area 2. Choose **Schedule a visual test** 3. Enter the URL of the webpage you want to visually test ![Add codeless visual test](https://testingbot.com/assets/support/visual/codeless/edit-758f0f36126b9487b7708d6b364225783a3714edbf64cba9c2077464ba758194.webp) ## Select Browsers You'll need to select one or more browsers before you can start the visual test. A maximum amount of 5 different browser configurations is possible per test. You'll be able to choose between various platforms and browsers. Each browser runs on the latest version that is available to the general public. For every browser configuration, you can specify a screen resolution and a geographical location. ### Available platforms - Windows 7 - Windows 8 - Windows 8.1 - Windows 10 - Windows 11 - macOS Tahoe - macOS Sequoia - macOS Sonoma - macOS Ventura - macOS Monterey - iOS Simulators - Android Emulators ### Available browsers - Chrome + Mobile Chrome - Firefox - Safari + Mobile Safari - Microsoft Edge - Internet Explorer ## Schedule Visual Tests TestingBot allows you to specify a schedule when visual tests should run. You have various options to choose from: - Run once on a specific date and time - Run a visual test daily, at a specific time - Run the test on a weekly basis, on a specific number of days and a specific hour - Repeat the test every x minutes, on specific days and between specific hours - Use a [Cron Format](https://en.wikipedia.org/wiki/Cron) to specify when the test should run ![Schedule a codeless visual test](https://testingbot.com/assets/support/visual/codeless/schedule-b7e3d090ab158252ff773d5afddaff6e3e6004a457f8874ed5022d25e0eb9c92.webp) ## Alerting When a visual test fails, you can choose to be alerted immediately. ### Email Alert One or more e-mail addresses can be added to the alerting section of each test. As soon as a visual test fails, you'll be alerted through e-mail. The e-mail message will contain more details about what failed. The current screenshot will be shown, together with the baseline screenshot and a diff image (highlighting the differences between the baseline and the screenshot). ### SMS Alert You have the option to add your phone number, we will send you a text message when a visual test fails. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/visual-testing/automated # Automated Visual Testing TestingBot provides a Visual Regression Testing service which integrates directly with the following test frameworks: - [Appium](https://testingbot.com/support/visual-testing/automated/appium) - [Selenium](https://testingbot.com/support/visual-testing/automated/selenium) - [Playwright](https://testingbot.com/support/visual-testing/automated/playwright) - [Puppeteer](https://testingbot.com/support/visual-testing/automated/puppeteer) This visual testing functionality allows you to quickly and easily add visual checkpoints to your existing or newly created automated tests. TestingBot will take a screenshot of your website or app and perform a pixel-by-pixel scan, comparing it to a previous golden image. For every visual check you perform in your test, TestingBot will do the comparison and report back the results. Each result will be visible in the [TestingBot member area](https://testingbot.com/members), allowing you to compare the screenshot with the baseline. Your test scripts will be able to call a custom TestingBot command and receive back the comparison results instantly. This means you can have your tests instantly verify the UI results of your website and mobile apps. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/visual-testing/automated/selenium # Selenium Visual Testing Selenium is a browser automation framework, designed to automate browsers. While it is used for running functional cross browser tests, it lacks the ability to compare the UI of webpages across various browsers. TestingBot offers a visual regression testing feature which integrates in your existing or new Selenium WebDriver scripts. By adding one or more visual checks to your Selenium test, you can easily do UI comparison checks and detect visual regressions. ## Setup TestingBot introduces a new Javascript Executor command which you can use in your Selenium tests, called `tb:visual.snapshot`. Simply pass a unique identifier to this function and TestingBot will compare the current UI of the webpage with a previous baseline/golden image. When you run this command for the first time, a screenshot will be taken from the page and will be marked as the baseline. Any future calls with the same identifier will trigger a pixel comparison on TestingBot's end. If TestingBot detects the number of different pixels to be higher than the supplied threshold, the call will return a response indicating the UI has changed. To add the command to your Selenium tests, please see the example below, where we save a screenshot of the current page as `uniqueIdentifier`. [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) [C#](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) ```ruby driver.execute_script('tb:visual.snapshot=uniqueIdentifier') ``` ```java ((JavascriptExecutor)driver).executeScript("tb:visual.snapshot=uniqueIdentifier"); ``` ```php $web_driver->executeScript('tb:visual.snapshot=uniqueIdentifier'); ``` ```python driver.execute_script('tb:visual.snapshot=uniqueIdentifier') ``` ```javascript await driver.executeScript('tb:visual.snapshot=uniqueIdentifier') ``` ```csharp ((IJavaScriptExecutor)driver).ExecuteScript("tb:visual.snapshot=uniqueIdentifier"); ``` The identifier passed to `tb:visual.snapshot` is unique per browser vendor, not per browser version. TestingBot considers browser versions to generate identical UI's, whereas different browser vendors might generate slightly different UI's (for example Safari vs Chrome). Make sure you're always comparing against the same [screen-resolution](https://testingbot.com/support/web-automate/selenium/test-options#screenresolution) as the screen-resolution that was taken during the baseline capture. ## Visual Comparison Response Whenever you run a visual comparison during a Selenium test, you will receive back a response in less than 2 seconds. For each visual test, you will receive back a response indicating: - Does the screenshot match the baseline? - What are the number of pixels that are different between the two snapshots? - The URL to the result in the TestingBot dashboard - The visualId and runId of the visual test ```json { "match": false, "pixelDifference": 341575, "url": "https://testingbot.com/members/visual/:visualId/runs/:runId", "visualId": ":visualId", "runId": ":runId" } ``` Please see the example below on how to check whether a visual change was detected: [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) [C#](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) ```ruby require 'json' visual_response = driver.execute_script('tb:visual.snapshot=uniqueIdentifier') visual_json = JSON.parse(visual_response) visual_test_passed = visual_json["match"] == true ``` ```java String jsonString = (String) ((JavascriptExecutor) driver).executeScript("tb:visual.snapshot=uniqueIdentifier"); ObjectMapper objectMapper = new ObjectMapper(); try { JsonNode jsonNode = objectMapper.readTree(jsonString); Boolean visualTestPassed = jsonNode.get("match").asBoolean(); } catch (Exception e) { e.printStackTrace(); } ``` ```php $visual_response = $web_driver->executeScript('tb:visual.snapshot=uniqueIdentifier'); $visual_json = json_decode($visual_response, true); $visual_test_passed = $visual_json["match"] === true; ``` ```python json_string = driver.execute_script("tb:visual.snapshot=uniqueIdentifier") # Decode the JSON response using the json module json_object = json.loads(json_string) # Check if the 'match' key is true visual_test_passed = json_object.get('match', False) ``` ```javascript const visual_response = await driver.executeScript('tb:visual.snapshot=uniqueIdentifier') const json_response = JSON.parse(visual_response) const visual_test_passed = json_response['match'] === true ``` ```csharp var jsonScript = "tb:visual.snapshot=uniqueIdentifier"; var jsonResult = (string)driver.ExecuteScript(jsonScript); var jsonObject = JObject.Parse(jsonResult); var visualTestPassed = (bool?)jsonObject["match"] == true; ``` ## Visual Baseline The first time you'll run the compare command, TestingBot will automatically mark the screenshot as the baseline. If you want to change the baseline after the first run, you can use the `tb:visual.baseline` command to take a new screenshot and set it as the baseline (golden image). You might want to use it to update the baseline when your webpage has changed its content or layout. Please see the example below on how to set the baseline to a new screenshot: [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) [C#](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) ```ruby driver.execute_script('tb:visual.baseline=uniqueIdentifier') ``` ```java ((JavascriptExecutor) driver).executeScript("tb:visual.baseline=uniqueIdentifier"); ``` ```php $web_driver->executeScript('tb:visual.baseline=uniqueIdentifier'); ``` ```python driver.execute_script("tb:visual.baseline=uniqueIdentifier") ``` ```javascript await driver.executeScript('tb:visual.baseline=uniqueIdentifier') ``` ```csharp driver.ExecuteScript("tb:visual.baseline=uniqueIdentifier"); ``` ## Visual Testing Options TestingBot provides various options to customize the visual comparison tests. Options can be passed with the `tb:visual.snapshot` command, for example: ```javascript const visualSnapshotJsonCmd = JSON.stringify({ name: 'testing123', options: { threshold: 0.3 } }) await driver.executeScript(`tb:visual.snapshot=${visualSnapshotJsonCmd}`) ``` Option Name | Default Option Value | Description || `threshold` | `0.1` | This defines the color difference threshold (from 0 to 1). The smaller the number, the more precise the comparison will be. | | `antialiasing` | `true` | When this is set to `true`, TestingBot will not count antialiased pixels to the diff of the snapshot. | | `ignoreRegions` | `[]` | Regions to ignore, defined in pixel coordinates. Needs to be an array with these objects: ```json [{ "x1": number, "y1": number, "x2": number, "y2": number }] ``` | | `ignoreSelectors` | `[]` | Pass an array of CSS Selectors to ignore these DOM elements during the visual check. ```json ["body div.test", "#sidebar"] ``` | | `selector` | `-` | Pass a CSS selector to take a screenshot of this specific element only. If this is not specified, TestingBot will take a screenshot of the viewport. ```json "body div.test" ``` | | `fullpage` | `false` | If you'd like to take a screenshot of the entire page, specify `true`. TestingBot will capture the entire page, instead of only the viewport. This feature is currently only available for these browsers: Chrome, Microsoft Edge and Firefox. | [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) [C#](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) ```ruby visual_snapshot_json_cmd = { name: 'testing123', options: { threshold: 0.3 } }.to_json driver.execute_script("tb:visual.snapshot=#{visual_snapshot_json_cmd}") ``` ```java String visualSnapshotJsonCmd = "{\"name\": \"testing123\", \"options\": {\"threshold\": 0.3}}"; ((JavascriptExecutor) driver).executeScript("tb:visual.snapshot=" + visualSnapshotJsonCmd); ``` ```php $visualSnapshotJsonCmd = json_encode([ 'name' => 'testing123', 'options' => [ 'threshold' => 0.3 ] ]); $driver->executeScript("tb:visual.snapshot={$visualSnapshotJsonCmd}"); ``` ```python visual_snapshot_json_cmd = json.dumps({ "name": "testing123", "options": { "threshold": 0.3 } }) driver.execute_script(f"tb:visual.snapshot={visual_snapshot_json_cmd}") ``` ```javascript const visualSnapshotJsonCmd = JSON.stringify({ name: 'testing123', options: { threshold: 0.3 } }) await driver.executeScript(`tb:visual.snapshot=${visualSnapshotJsonCmd}`) ``` ```csharp var visualSnapshotJsonCmd = "{\"name\": \"testing123\", \"options\": {\"threshold\": 0.3}}"; ((IJavaScriptExecutor)driver).ExecuteScript($"tb:visual.snapshot={visualSnapshotJsonCmd}"); ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/visual-testing/automated/appium # Appium Visual Testing Appium is a mobile test automation framework, built to run web and mobile app tests on mobile simulators and physical devices. Visual testing is currently not yet supported by Appium. TestingBot has built a visual regression testing solution for Appium, which you can use without installing any plugins or custom code. By adding one or more visual checks to your Appium test, you can quickly add visual checkpoints to your Appium test scripts. TestingBot will let you know if a snapshot of your current app looks identical to the baseline that you've set. ## Setup TestingBot added a new Javascript Executor command which can be used in Appium tests, called `tb:visual.snapshot`. By passing a unique identifier with this command, TestingBot will compare the initial screenshot taken with this identifier to the current screenshot. TestingBot will perform a pixel comparison scan, comparing the screenshot of your mobile app or website with the original golden image (baseline). If TestingBot detects the number of different pixels to be higher than the supplied threshold, the test will be marked as failed. To add one or more visual checks to your Appium tests, please see the example below, where we save a screenshot of the current mobile app/website state as `uniqueIdentifier`. [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) [C#](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) ```ruby driver.execute_script('tb:visual.snapshot=uniqueIdentifier') ``` ```java ((JavascriptExecutor)driver).executeScript("tb:visual.snapshot=uniqueIdentifier"); ``` ```php $web_driver->executeScript('tb:visual.snapshot=uniqueIdentifier'); ``` ```python driver.execute_script('tb:visual.snapshot=uniqueIdentifier') ``` ```javascript await driver.executeScript('tb:visual.snapshot=uniqueIdentifier') ``` ```csharp ((IJavaScriptExecutor)driver).ExecuteScript("tb:visual.snapshot=uniqueIdentifier"); ``` ## Visual Comparison Response During your Appium test, you can perform one or more visual comparisons and receive back a response in less than 2 seconds. For each visual test, you will receive back a response indicating: - Does the screenshot match the baseline? - What are the number of pixels that are different between the two snapshots? - The URL to the result in the TestingBot dashboard - The visualId and runId of the visual test ```json { "match": false, "pixelDifference": 341575, "url": "https://testingbot.com/members/visual/:visualId/runs/:runId", "visualId": ":visualId", "runId": ":runId" } ``` Please see the example below on how to check whether a visual change was detected: [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) [C#](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) ```ruby require 'json' visual_response = driver.execute_script('tb:visual.snapshot=uniqueIdentifier') visual_json = JSON.parse(visual_response) visual_test_passed = visual_json["match"] == true ``` ```java String jsonString = (String) ((JavascriptExecutor) driver).executeScript("tb:visual.snapshot=uniqueIdentifier"); ObjectMapper objectMapper = new ObjectMapper(); try { JsonNode jsonNode = objectMapper.readTree(jsonString); Boolean visualTestPassed = jsonNode.get("match").asBoolean(); } catch (Exception e) { e.printStackTrace(); } ``` ```php $visual_response = $web_driver->executeScript('tb:visual.snapshot=uniqueIdentifier'); $visual_json = json_decode($visual_response, true); $visual_test_passed = $visual_json["match"] === true; ``` ```python json_string = driver.execute_script("tb:visual.snapshot=uniqueIdentifier") # Decode the JSON response using the json module json_object = json.loads(json_string) # Check if the 'match' key is true visual_test_passed = json_object.get('match', False) ``` ```javascript const visual_response = await driver.executeScript('tb:visual.snapshot=uniqueIdentifier') const json_response = JSON.parse(visual_response) const visual_test_passed = json_response['match'] === true ``` ```csharp var jsonScript = "tb:visual.snapshot=uniqueIdentifier"; var jsonResult = (string)driver.ExecuteScript(jsonScript); var jsonObject = JObject.Parse(jsonResult); var visualTestPassed = (bool?)jsonObject["match"] == true; ``` ## Visual Baseline The first time that you run the compare command with a given identifier, TestingBot will automatically mark the screenshot as the baseline. If you want to update the baseline after it has already been saved, you can use the `tb:visual.baseline` command to take a new screenshot and set it as the new baseline image. Please see the example below on how to update the baseline for a given identifier: [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) [C#](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) ```ruby driver.execute_script('tb:visual.baseline=uniqueIdentifier') ``` ```java ((JavascriptExecutor) driver).executeScript("tb:visual.baseline=uniqueIdentifier"); ``` ```php $web_driver->executeScript('tb:visual.baseline=uniqueIdentifier'); ``` ```python driver.execute_script("tb:visual.baseline=uniqueIdentifier") ``` ```javascript await driver.executeScript('tb:visual.baseline=uniqueIdentifier') ``` ```csharp driver.ExecuteScript("tb:visual.baseline=uniqueIdentifier"); ``` ## Visual Testing Options TestingBot offers various options to customize the visual comparison tests. Options can be passed with the `tb:visual.snapshot` command: ```javascript const visualSnapshotJsonCmd = JSON.stringify({ name: 'testing123', options: { threshold: 0.3 } }) driver.executeScript(`tb:visual.snapshot=${visualSnapshotJsonCmd}`) ``` Option Name | Default Option Value | Description || `threshold` | `0.1` | This defines the color difference threshold (from 0 to 1). The smaller the number, the more precise the comparison will be. | | `antialiasing` | `true` | When this is set to `true`, TestingBot will not count antialiased pixels to the diff of the snapshot. | | `ignoreRegions` | `[]` | Regions to ignore, defined in pixel coordinates. Needs to be an array with these objects: ```json [{ "x1": number, "y1": number, "x2": number, "y2": number }] ``` | | `ignoreSelectors` Web Testing Only | `[]` | Pass an array of CSS Selectors to ignore these DOM elements during the visual check. ```json ["body div.test", "#sidebar"] ``` | | `selector` Web Testing Only | `-` | Pass a CSS selector to take a screenshot of this specific element only. If this is not specified, TestingBot will take a screenshot of the viewport. ```json "body div.test" ``` | [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) [C#](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) ```ruby visual_snapshot_json_cmd = { name: 'testing123', options: { threshold: 0.3 } }.to_json driver.execute_script("tb:visual.snapshot=#{visual_snapshot_json_cmd}") ``` ```java String visualSnapshotJsonCmd = "{\"name\": \"testing123\", \"options\": {\"threshold\": 0.3}}"; ((JavascriptExecutor) driver).executeScript("tb:visual.snapshot=" + visualSnapshotJsonCmd); ``` ```php $visualSnapshotJsonCmd = json_encode([ 'name' => 'testing123', 'options' => [ 'threshold' => 0.3 ] ]); $driver->executeScript("tb:visual.snapshot={$visualSnapshotJsonCmd}"); ``` ```python visual_snapshot_json_cmd = json.dumps({ "name": "testing123", "options": { "threshold": 0.3 } }) driver.execute_script(f"tb:visual.snapshot={visual_snapshot_json_cmd}") ``` ```javascript const visualSnapshotJsonCmd = JSON.stringify({ name: 'testing123', options: { threshold: 0.3 } }) await driver.executeScript(`tb:visual.snapshot=${visualSnapshotJsonCmd}`) ``` ```csharp var visualSnapshotJsonCmd = "{\"name\": \"testing123\", \"options\": {\"threshold\": 0.3}}"; ((IJavaScriptExecutor)driver).ExecuteScript($"tb:visual.snapshot={visualSnapshotJsonCmd}"); ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/visual-testing/automated/puppeteer # Puppeteer Visual Testing Using TestingBot's Puppeteer Visual Testing command, you will be able to take screenshots during your Puppeteer tests and compare these with baselines. You will see these results in the TestingBot member area, with additional details such as match percentage, a diff of the image and more useful information. For each screenshot taken during your Puppeteer test, TestingBot will perform a pixel-by-pixel comparison with the baseline image. If a mismatch is detected, your test will fail. ## Setup TestingBot added a new `page.evaluate` command which can be used in your Puppeteer tests. Simply pass a ``testingbot_executor: ${JSON.stringify({action: 'visual.snapshot', arguments: { name: 'uniqueIdentifier' }})}`` argument to have TestingBot take and compare the screenshot. TestingBot will perform a pixel comparison scan, comparing the screenshot of your website with the original golden image (baseline). If TestingBot detects the number of different pixels to be higher than the supplied threshold, the test will be marked as failed. ```javascript const results = await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'visual.snapshot', arguments: { name: 'uniqueIdentifier' }})}`) ``` In the example above, the screenshot will be saved as `uniqueIdentifier`. Any future calls will compare the screenshot with the screenshot that was saved the first time this command was run. ## Visual Comparison Response Every time you run a `visual.snapshot` command, TestingBot will return back the visual comparison result in less than 2 seconds. The response will contain the following information: - Does the screenshot match the baseline? - What are the number of pixels that are different between the two snapshots? - The URL to the result in the TestingBot dashboard - The visualId and runId of the visual test ```json { "match": false, "pixelDifference": 341575, "url": "https://testingbot.com/members/visual/:visualId/runs/:runId", "visualId": ":visualId", "runId": ":runId" } ``` Please see the example below on how to check whether a visual change was detected: ```javascript const visual_response = await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'visual.snapshot', arguments: { name: 'uniqueIdentifier' }})}`) const json_response = JSON.parse(visual_response) const visual_test_passed = json_response['match'] === true ``` ## Visual Baseline To set a new baseline for your snapshots, you can use the following command: ```javascript const visual_response = await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'visual.baseline', arguments: { name: 'uniqueIdentifier' }})}`) ``` ## Visual Testing Options TestingBot offers various options to customize the visual comparison tests. Options can be passed with the `visual.snapshot` command: ```javascript const visualSnapshotJsonCmd = JSON.stringify({ name: 'testing123', options: { threshold: 0.3 } }) const visual_response = await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'visual.snapshot', arguments: visualSnapshotJsonCmd })}`) ``` Option Name | Default Option Value | Description || `threshold` | `0.1` | This defines the color difference threshold (from 0 to 1). The smaller the number, the more precise the comparison will be. | | `antialiasing` | `true` | When this is set to `true`, TestingBot will not count antialiased pixels to the diff of the snapshot. | | `ignoreRegions` | `[]` | Regions to ignore, defined in pixel coordinates. Needs to be an array with these objects: ```json [{ "x1": number, "y1": number, "x2": number, "y2": number }] ``` | | `ignoreSelectors` | `[]` | Pass an array of CSS Selectors to ignore these DOM elements during the visual check. ```json ["body div.test", "#sidebar"] ``` | | `selector` | `-` | Pass a CSS selector to take a screenshot of this specific element only. If this is not specified, TestingBot will take a screenshot of the viewport. ```json "body div.test" ``` | ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/visual-testing/automated/playwright # Playwright Visual Testing While [Playwright Visual Regression Testing](https://testingbot.com/support/web-automate/playwright/visual-regression-testing) is already included with Playwright Test, TestingBot now offers a visual comparison feature for all automated tests. Using TestingBot's visual command, you will be able to take screenshots during your Playwright tests and compare these with baselines. You will see these results in the TestingBot member area, with additional details such as match percentage, a diff of the image and more useful information. ## Setup TestingBot added a new `page.evaluate` command which can be used in your Playwright tests. Simply pass a ``testingbot_executor: ${JSON.stringify({action: 'visual.snapshot', arguments: { name: 'uniqueIdentifier' }})}`` argument to have TestingBot take and compare the screenshot. TestingBot will perform a pixel comparison scan, comparing the screenshot of your website with the original golden image (baseline). If TestingBot detects the number of different pixels to be higher than the supplied threshold, the test will be marked as failed. ```javascript const results = await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'visual.snapshot', arguments: { name: 'uniqueIdentifier' }})}`) ``` In the example above, the screenshot will be saved as `uniqueIdentifier`. Any future calls will compare the screenshot with the screenshot that was saved the first time this command was run. ### Full Playwright Example Below is a full example on how to run a visual Playwright test on TestingBot's browser cloud. ```javascript const playwright = require('playwright-core'); (async () => { const browser = await playwright.chromium.connectOverCDP('wss://cloud.testingbot.com/?key=api_key&secret=api_secret&browserName=chrome&browserVersion=latest') const page = await browser.newPage() await page.goto('https://testingbot.com') const results = await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'visual.snapshot', arguments: { name: 'uniqueIdentifier' }})}`) console.log(results) await browser.close() })(); ``` Visual testing with Playwright on TestingBot's browser cloud currently only works with Chromium-based browsers (Chrome, Microsoft Edge) through `connectOverCDP`. ## Visual Comparison Response Every time you run a `visual.snapshot` command, TestingBot will return back the visual comparison result in less than 2 seconds. The response will contain the following information: - Does the screenshot match the baseline? - What are the number of pixels that are different between the two snapshots? - The URL to the result in the TestingBot dashboard - The visualId and runId of the visual test ```json { "match": false, "pixelDifference": 341575, "url": "https://testingbot.com/members/visual/:visualId/runs/:runId", "visualId": ":visualId", "runId": ":runId" } ``` Please see the example below on how to check whether a visual change was detected: ```javascript const visual_response = await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'visual.snapshot', arguments: { name: 'uniqueIdentifier' }})}`) const json_response = JSON.parse(visual_response) const visual_test_passed = json_response['match'] === true ``` ## Visual Baseline To set a new baseline for your snapshots, you can use the following command: ```javascript const visual_response = await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'visual.baseline', arguments: { name: 'uniqueIdentifier' }})}`) ``` ## Visual Testing Options TestingBot offers various options to customize the visual comparison tests. Options can be passed with the `visual.snapshot` command: ```javascript const visualSnapshotJsonCmd = JSON.stringify({ name: 'testing123', options: { threshold: 0.3 } }) const visual_response = await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'visual.snapshot', arguments: visualSnapshotJsonCmd })}`) ``` Option Name | Default Option Value | Description || `threshold` | `0.1` | This defines the color difference threshold (from 0 to 1). The smaller the number, the more precise the comparison will be. | | `antialiasing` | `true` | When this is set to `true`, TestingBot will not count antialiased pixels to the diff of the snapshot. | | `ignoreRegions` | `[]` | Regions to ignore, defined in pixel coordinates. Needs to be an array with these objects: ```json [{ "x1": number, "y1": number, "x2": number, "y2": number }] ``` | | `ignoreSelectors` | `[]` | Pass an array of CSS Selectors to ignore these DOM elements during the visual check. ```json ["body div.test", "#sidebar"] ``` | | `selector` | `-` | Pass a CSS selector to take a screenshot of this specific element only. If this is not specified, TestingBot will take a screenshot of the viewport. ```json "body div.test" ``` | ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations # TestingBot Integrations TestingBot integrates with the CI/CD systems, issue trackers and testing tools you already use. Browse the integrations below by category. ## Issue & Project Management Push results, file bugs and get notified in the tools your team already uses. ![Jira](https://testingbot.com/assets/integrations/jira-63eb4095ef566b1fe712e84f18e56b37ea706defb2b21450d93f1304c9886fa7.svg) ### [Jira](https://testingbot.com/support/integrations/jira) Create issues with test meta-data ![GitHub](https://testingbot.com/assets/integrations/github-d03b3ecd2162b559b5160a1e89db53b782a287ae0402f80469c7d5be1e54646b.svg) ### [GitHub](https://testingbot.com/support/integrations/github-issue-tracker) Create issues with test meta-data ![Azure DevOps](https://testingbot.com/assets/integrations/azure-devops-f5023054e94afd0bc317b12821d90e51ab039b9cdabd7ab875a8afba875b8430.svg) ### [Azure DevOps](https://testingbot.com/support/integrations/azure-devops-issue-tracker) Create work items with test meta-data ![Trello](https://testingbot.com/assets/integrations/trello-38611a1d274409477be6bd296f5bcd470c5285b06f24ecfe71bac7dd5112c9dd.svg) ### [Trello](https://testingbot.com/support/integrations/trello-issue-tracker) Create cards from test failures ![Asana](https://testingbot.com/assets/integrations/asana-9c0452f90913962626d5b2057d62f61e969057dc2eabb1964524acf7977ab76c.svg) ### [Asana](https://testingbot.com/support/integrations/asana-issue-tracker) Create tasks from test failures ![Linear](https://testingbot.com/assets/integrations/linear-35f1036d530044281bff96db8adac28994040fe1f2a8ecf032aca7f011d26690.svg) ### [Linear](https://testingbot.com/support/integrations/linear-issue-tracker) Create issues from test failures ![Bugsnag](https://testingbot.com/assets/integrations/bugsnag-3e0b2bebfb3c599b8d986dda1bd59712ae70de4a9450b7522a698ad786ff6e62.svg) ### [Bugsnag](https://testingbot.com/support/integrations/bugsnag) Create issues with screenshots ![Slack](https://testingbot.com/assets/integrations/slack-ba22a02abd583ab16127eed596daaa8e504ec2a1ab3c4922e3e8d19dcf052e65.svg) ### [Slack](https://testingbot.com/support/integrations/slack) Get notified when tests finish or fail ![Zebrunner](https://testingbot.com/assets/integrations/zebrunner-51fe5d07743785e67d69f0c97a8ae327e89a01545a54662e5cf1cf3c24601bf4.svg) ### [Zebrunner](https://testingbot.com/support/integrations/zebrunner) Smart analysis of test results ![Webhooks](https://testingbot.com/assets/integrations/webhooks-a941c2149b97f9fdfe154a2aad83f41206612cf0be3e61a6c77418914e32a0e6.svg) ### [Webhooks](https://testingbot.com/support/integrations/webhooks) Receive an HTTP callback on test events ## CI/CD Run your tests automatically on every commit, build or deploy. ![AWS CodePipeline](https://testingbot.com/assets/integrations/aws-37fe010e66887d20cd831130e90da33a0ec731b5dd87fbba8adc933db17b0709.webp) ### [AWS CodePipeline](https://testingbot.com/support/integrations/ci-cd/aws-codepipeline) CodeBuild buildspec with tunnel ![Azure DevOps (VSTS)](https://testingbot.com/assets/integrations/devops-7de50ed101385e09ac3fb0b1110a601e229282512a1b25460fdaedd22ef244bb.webp) ### [Azure DevOps (VSTS)](https://testingbot.com/support/integrations/ci-cd/azure) Integrate with VSTS ![Bamboo](https://testingbot.com/assets/integrations/bamboo-c8fe09a3cb0b74d6055e3aa6607d77720bfe29ff487b8e3cf825138adf634346.svg) ### [Bamboo](https://testingbot.com/support/integrations/ci-cd/bamboo) Atlassian Bamboo plugin ![Bitbucket Pipelines](https://testingbot.com/assets/integrations/bitbucket-02aabcd549221726e052f2ee19c8959fbb218160ad7f7c31f6e6b96e7824a42c.webp) ### [Bitbucket Pipelines](https://testingbot.com/support/integrations/ci-cd/bitbucket) Bitbucket Pipelines integration ![Bitrise](https://testingbot.com/assets/integrations/bitrise-d122449f8ba4609306287a7080cef27601f50d50b7bdf23811751de4d5ea41d6.svg) ### [Bitrise](https://testingbot.com/support/integrations/ci-cd/bitrise) Mobile app CI/CD integration ![Buildkite](https://testingbot.com/assets/integrations/buildkite-d573ce24dc0bb8ce8df123e751563dd8ee85b93dcdaba4a49d72ff0a678d34f6.svg) ### [Buildkite](https://testingbot.com/support/integrations/ci-cd/buildkite) Tunnel, statuses and annotations ![CircleCI](https://testingbot.com/assets/integrations/circleci-ede029654ff39b60342e4a5bba8fc9a3a49305cc38bda215b7cf8d07e85e0543.webp) ### [CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci) Integrate with CircleCI ![GitHub Actions](https://testingbot.com/assets/integrations/github-actions-8436242ad9e15381f84f1a98a10dddce15611087f121d6305782e211942b7842.webp) ### [GitHub Actions](https://testingbot.com/support/integrations/ci-cd/github-actions) Launch tests from workflows ![GitHub PR Checks](https://testingbot.com/assets/integrations/github-d03b3ecd2162b559b5160a1e89db53b782a287ae0402f80469c7d5be1e54646b.svg) ### [GitHub PR Checks](https://testingbot.com/support/integrations/ci-cd/github-pr-checks) Status checks on pull requests ![GitLab CI](https://testingbot.com/assets/integrations/gitlab-e4c0b6db07d07bc0a400f5606e258f4dc8a51b778e97bbfa51d1d531f3cb049a.webp) ### [GitLab CI](https://testingbot.com/support/integrations/ci-cd/gitlab) Integrate with GitLab CI ![Gradle](https://testingbot.com/assets/integrations/gradle-a7d0123cadbbf7d2bf598be31e0b4cf9f035c9924f988603d5b8a533385ef352.svg) ### [Gradle](https://testingbot.com/support/integrations/ci-cd/gradle) Espresso tests from your build ![Jenkins CI](https://testingbot.com/assets/integrations/jenkins-e586a1c4af833e4c8d38d7b392a4e84bd193f30cae9cf8c9f198a58ec943a1ff.webp) ### [Jenkins CI](https://testingbot.com/support/integrations/ci-cd/jenkins) Pipeline and tunnel support ![TeamCity CI](https://testingbot.com/assets/integrations/teamcity-7e75cdb835903a4983ad276c41d95e569ceac5be486e96ec1f955029ce5bb319.svg) ### [TeamCity CI](https://testingbot.com/support/integrations/ci-cd/teamcity) Plugin for JetBrains TeamCity ![Travis CI](https://testingbot.com/assets/integrations/travis-f3a8872deab0dd6f4c13686fd8255cb07c426309090e335a9198312b49c43232.webp) ### [Travis CI](https://testingbot.com/support/integrations/ci-cd/travis-ci) Integrate with Travis CI ## Record & Playback Connect low-code and record-and-playback tools to TestingBot. ![Katalon Studio](https://testingbot.com/assets/integrations/katalon-497d6adbf9a9bb1ed8b5c9b9d6b8ded1b45dfb63a3386b572a0401515e4377ca.webp) ### [Katalon Studio](https://testingbot.com/support/integrations/katalon-studio) Record and run on TestingBot ![Ranorex Studio](https://testingbot.com/assets/integrations/ranorex-180a3aafc4fbb4b4d78a0d99af328c33256d85e5d7fad77c0a75a67e40ddcb69.webp) ### [Ranorex Studio](https://testingbot.com/support/integrations/ranorex) Record and run tests ![Cerberus](https://testingbot.com/assets/integrations/cerberus-21bf19476562da6ec6b57ca5acdc63bada4fa7093018801ecb2a83b9c5edb98e.webp) ### [Cerberus](https://testingbot.com/support/integrations/cerberus) Run Cerberus tests ![Tricentis Tosca](https://testingbot.com/assets/integrations/tricentis-tosca-89d025a6a71944c379e9ef745466c8b65c6af8141d72350e2bc6c33ab85b6da8.webp) ### [Tricentis Tosca](https://testingbot.com/support/integrations/tricentis-tosca) Run Tosca tests ![QMetry](https://testingbot.com/assets/integrations/qmetry-0647488fc2248e9e180e384e614b8cdbd1e8e4cf5360ab450d65a512490b0405.svg) ### [QMetry](https://testingbot.com/support/integrations/qmetry) Codeless test automation ## App Distribution Pull app builds straight from your distribution pipeline. ![App Center](https://testingbot.com/assets/integrations/app-center-icon-2c5c0d6c296fd04d1d3d0109db02dde20bb93a5ee7a9616e7805f27d6ec6503a.svg) ### [App Center](https://testingbot.com/support/integrations/appcenter) Pull builds from App Center ![Firebase App Distribution](https://testingbot.com/assets/integrations/firebase-a7d0366e8d92c33287523d31685c80a3472833f041aacafceb5a9a93f8a45483.svg) ### [Firebase App Distribution](https://testingbot.com/support/integrations/firebase-app-distribution) Pull builds from Firebase ![Fastlane](https://testingbot.com/assets/integrations/fastlane-a14bb29fa79676049d86e42c5afd589946be86729c1493bf558240c697811fb0.png) ### [Fastlane](https://testingbot.com/support/integrations/fastlane) Automate build and upload ## Other Additional tooling that works with TestingBot. ![Appium Inspector](https://testingbot.com/assets/integrations/appium-75e88eb18ca3b2ce63d641547ce06398c8ecd971ed17187c9134c8d205465f18.svg) ### [Appium Inspector](https://testingbot.com/support/integrations/appium-desktop) Inspect app elements in a live view ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/appcenter # Visual Studio App Center **Visual Studio App Center was retired by Microsoft on March 31, 2025** and is no longer operational. The integration described on this page is kept for historical reference. For current mobile-build CI options, see [Bitrise](https://testingbot.com/support/integrations/ci-cd/bitrise), [GitHub Actions](https://testingbot.com/support/integrations/ci-cd/github-actions), or [GitLab CI](https://testingbot.com/support/integrations/ci-cd/gitlab). [Visual Studio App Center](https://learn.microsoft.com/en-us/appcenter/retirement) was a Microsoft product that built mobile apps in the cloud — connect a repository and App Center would build the app on every commit. This integration allowed you to test builds generated by App Center on our mobile device cloud. In this guide we'll show you: - How to connect to the Visual Studio App Center API. - Fetch the public link to the latest build. - Upload the build to [TestingBot Storage](https://testingbot.com/support/app-automate/help/upload) for testing. ## Retrieve API Token To get started, you'll need to log into [Visual Studio App Center](https://appcenter.ms/) and generate an API key. - Sign into your App Center account. - Go to **Account Settings** and choose **User API Tokens**. - Click the **New API Token** button. - Add a description for your token, for example **TestingBot-Token**. - Give the token **Read Only** access. - Generate and copy the API Token. ![Visual Studio App Center API Token](https://testingbot.com/assets/support/appcenter/token-850b6e80bb9e0adc12f0798d19539bdb7c3837ff34847d8a8d0808397037b90b.png) ## List All App Projects Now that we have an API token, we can use it to fetch all app projects: ```bash curl -sX GET "https://api.appcenter.ms/v0.1/apps" \ -H "Content-Type: application/json" \ -H "X-Api-Token: [YOUR-API-TOKEN]" ``` The response should be similar to this: ```json [{ "id": "9c22426b-e23e-4389-9f4a-08ec2eb9e656", "app_secret": "3da37025-4548-4abe-8979-c0f097816e51", "description": null, "display_name": "testingbot-demo", "name": "testingbot-demo", "os": "Android", "platform": "React-Native", "origin": "appcenter", "icon_url": null, "created_at": "2020-08-19T08:39:21.000Z", "updated_at": "2020-08-19T08:39:21.000Z", "release_type": null, "owner": { "id": "4a05e941-f5be-4cfe-8b25-3887a768578f", "avatar_url": null, "display_name": "TestingBot", "email": "...", "name": "testingbot", "type": "user" }, "azure_subscription": null, "member_permissions": ["manager"] }, { ... }] ``` ## Fetch build info from API Now we can fetch the build information from the App Center API. We'll need the **name** and **owner.name** from the previous response: If you want to fetch the build of the latest release, please use: ```bash curl -sX GET "https://api.appcenter.ms/v0.1/apps/[owner.name]/[name]/releases/latest" \ -H "Content-Type: application/json" \ -H "X-Api-Token: [YOUR-API-TOKEN]" ``` If you are looking to fetch the latest build (not necessarily released yet), please use: ```bash curl -sX GET "https://api.appcenter.ms/v0.1/apps/[owner.name]/[name]/branches/master/builds" \ -H "Content-Type: application/json" \ -H "X-Api-Token: [YOUR-API-TOKEN]" ``` Then copy the build id and use: ```bash curl -sX GET "https://api.appcenter.ms/v0.1/apps/[owner.name]/[name]/builds/[build-id]/downloads/build" \ -H "Content-Type: application/json" \ -H "X-Api-Token: [YOUR-API-TOKEN]" ``` The response should contain a URL which points to the build, similar to this example: ```json {"uri": "https://build.appcenter.ms/v0.1/public/apps/9c22426b-e23e-4389-9f4a-oefkoe/downloads?token=6663fc99b13e70dsfds856c8a302e9d7398fkjecfc72b0c133bcf4035"} ``` ## Upload build Now that you've retrieved the build URL, you can upload this build to [TestingBot Storage](https://testingbot.com/support/app-automate/help/upload). ```bash curl -X POST "https://api.testingbot.com/v1/storage" \ -u API_KEY:API_SECRET -d "url=[BUILD-URL]" ``` This will return a unique URL `{"app_url": "tb://..."}` which you can use with your tests. Simply add the `tb://...` as an **app** capability to run your tests: [more information](https://testingbot.com/support/app-automate/help/upload). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/appium-desktop # Appium Inspector With [Appium Inspector](https://github.com/appium/appium-inspector) (previously known as Appium Desktop) you can visualize and inspect all the different elements in your mobile app. Locate XPaths and element identifiers by selecting the elements in the live view. Appium Inspector will show a live feed of the app. When you click an element it will show the possible locators for the specific element. This is a great help when developing or updating automated mobile app tests. TestingBot has a built-in integration with Appium Inspector. Simply paste your TestingBot `key` and `secret` in the TestingBot configuration setting (see below). ## Add TestingBot Select TestingBot from the **Cloud Providers** tab. A new TestingBot tab will appear. ![Appium Inspector with TestingBot](https://testingbot.com/assets/support/appium-desktop/overview-a331c33f7482b169896f85c37e8d06c2f628949cc57514005f70b1b6fb24e173.png) ## Configuration Next, you can fill in your TestingBot `KEY` and `SECRET`. This will be used to authenticate with our Appium grid. You can then fill in the desired capabilities, which are necessary to connect to a device in our grid. ![Appium Inspector with TestingBot](https://testingbot.com/assets/support/appium-desktop/capabilities-096bfa37f18a36513e6946f3e7be3251e91376dc525a5d0d8043e79266efeb2b.png) ## Inspect Once your session has started, you will see a live view of the device on the left and the possible locators for the selected element on the right. ![Appium Inspector with TestingBot](https://testingbot.com/assets/support/appium-desktop/inspect-d3fa2f26e7e45a864f38690862747d7c65e48eb90847794d8a91749f9cfc98f7.png) ## TestingBot Options To specify [TestingBot specific options](https://testingbot.com/support/app-automate/appium/options), please use `tb:options` as field name and `JSON Object` as field type. ![Appium Inspector with TestingBot](https://testingbot.com/assets/support/appium-desktop/options-a56030507f0b62f4765a145c14b9cfc17ed164a49d68d683a61aa9a4ebbc798b.png) ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/bugsnag # Bugsnag Integration Create issues in Bugsnag straight from TestingBot, during manual web and mobile app testing. Report any browser or device issue straight to Bugsnag, with screenshots and details. ## Features Adding this integration will allow you to share issues you've noticed during manual testing. - During manual testing on TestingBot, you can take a screenshot of the issue. - Annotate the screenshot to highlight the problem. - Post the screenshot, together with a description of the issue straight to your Bugsnag Project. - Team members on Bugsnag will see the issue listed in Bugsnag. ## Getting Started To get started, please follow these steps: - Log in to Bugsnag and click the **Settings** icon - Open the **Project Settings**. Your project settings will contain a **Notifier API key** , which you can copy and enter in the [Bugsnag Settings Form](https://testingbot.com/members/integrations/bugsnag) on TestingBot. ![Bugsnag Key](https://testingbot.com/assets/support/integrations/bugsnag/bugsnag_key-7174452f1756f55696f810b7ab7a8fd3c43056aab9b182d0a8f00e98f94737ac.webp) ## Report Issues - During a manual testing session, you can click the **Report Bug** option to take a live screenshot of the page. - You can annotate the screenshot to make the issue more clear, with a set of paint tools in the screenshot editor. - Finally, click the **Report a bug** button to see a list of services. Click Bugsnag to send the issue to your Bugsnag project. - Enter a title and a description of the issue. The screenshot will be included automatically. ![Bugsnag Issue](https://testingbot.com/assets/support/integrations/bugsnag/issue-3841d11b0c3e830f99f7c77482f1e09a84247d59710f22a77cf1b33b49d5ffa2.webp) ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/cerberus # Setting up TestingBot as the Executor Robot Cerberus Testing is a low-code test automation platform that supports testing Web, iOS and Android. This guide will help you with running tests on TestingBot from the Cerberus Testing Framework. Inside your Cerberus Testing instance you will need to set up TestingBot as your Test Executor. Please follow the steps below to set this up. 1. Click on **Run** and then click **Robot**. Next, click the button **Create new Robot**. ![Cerberus new Robot](https://testingbot.com/assets/support/integrations/cerberus/1-1c34e36c958ed167eb761cf229bfe90fbc631860acf820a158b0c16c421da764.webp) 2. Configure the Robot with the browser you want to test on, specify a version and operating system and give it a name, for example TestingBot-Robot. ![Cerberus TestingBot Robot](https://testingbot.com/assets/support/integrations/cerberus/2-c3d048385859c37a530b2a486bada42707ca6f1a11f8d9d53c8b213070a06712.webp) 3. Next, click **Executors** and specify these details: - **Host** : `hub.testingbot.com` - **Port** : `443` - **Host User** : your TestingBot key - **Host Password** : your TestingBot secret ![Cerberus TestingBot Robot](https://testingbot.com/assets/support/integrations/cerberus/3-b441e18a51ad28cb7efa767f985b6ac1c06200f47ea5136cfa43b4609a853f5e.webp) 4. Now it's time to run a test. Click **Run** and **Run Test Case** , then select a test case to run. ![Cerberus Run Test](https://testingbot.com/assets/support/integrations/cerberus/4-fb2061ae037a195cefe192f616b4ffab0de9e8a6451b69af04a5b96df799542b.webp) 5. Choose an environment, for example **PROD**. ![Cerberus Environment](https://testingbot.com/assets/support/integrations/cerberus/5-6f646e15df72c13324a2b8ff5eb62e75b4b713f94628df23c0e06defb04f2589.webp) 6. In the Robot settings, select the TestingBot-Robot you created earlier. ![Cerberus Robot Settings](https://testingbot.com/assets/support/integrations/cerberus/6-b9faea4d9562f9213492857d08f8723c9e5ef67bca2de7d7a1c6d291d7748340.webp) 7. Run the test, you should see a success message once the test has finished running. Test details, together with screenshots and a video will be available in your TestingBot member area. ![Cerberus Run test](https://testingbot.com/assets/support/integrations/cerberus/7-22cdac8272c56f09c7c8d1cff8665581e1e407d8e0eb2ee38588bc46ba90f25f.webp) ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/fastlane # Fastlane Integration [Fastlane](https://fastlane.tools) is the most popular open-source toolchain to automate building and releasing iOS and Android apps. Our [fastlane-plugin-testingbot](https://github.com/testingbot/testingbot-fastlane-plugin) plugin uploads your app builds to TestingBot Storage straight from a Fastlane lane, so they are ready for automated and manual testing on our real devices, emulators and simulators. Because the upload runs as a regular Fastlane action, you can chain it right after `gym` (iOS) or `gradle` (Android) in the same lane, no manual upload step and no extra scripts. ## Prerequisites - A [TestingBot account](https://testingbot.com/users/sign_up) with an API key and secret, which you can find in the [member area](https://testingbot.com/members/user/edit). - Fastlane installed in your project (see the [fastlane setup guide](https://docs.fastlane.tools/getting-started/ios/setup/)). - An app build to upload: `.apk` or `.aab` for Android, `.ipa` for iOS real devices, or a `.zip` for the iOS simulator. ## Install the plugin Add the plugin to your project with Fastlane's plugin command: ```bash fastlane add_plugin testingbot ``` This adds the plugin to your `fastlane/Pluginfile` and updates your `Gemfile` and `Gemfile.lock`. Commit these files so the plugin is installed automatically on your CI machines. ## Set your credentials The plugin authenticates with your TestingBot key and secret. The recommended way is to provide them as environment variables, so they are not stored in your `Fastfile`: ```bash export TESTINGBOT_KEY="your-api-key" export TESTINGBOT_SECRET="your-api-secret" ``` On CI services such as GitHub Actions, CircleCI or Bitrise, store `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` as secret/encrypted environment variables instead of hard-coding them. You can also pass the credentials inline with the `testingbot_key` and `testingbot_secret` parameters of the action, but environment variables are preferred. ## Upload your app The plugin adds the `upload_to_testingbot` action. In its simplest form, point it at a local build: ```ruby lane :upload do upload_to_testingbot( file_path: "./app/build/outputs/apk/debug/app-debug.apk" ) end ``` When you build in the same lane, you can drop `file_path` entirely. The action picks up the build that `gradle` or `gym` just produced: ```ruby # Android lane :android_build_and_upload do gradle(task: "assembleDebug") upload_to_testingbot end # iOS lane :ios_build_and_upload do gym(scheme: "MyApp") upload_to_testingbot end ``` You can also have TestingBot fetch the build from a remote URL, instead of uploading a local file: ```ruby upload_to_testingbot( remote_url: "https://example.com/builds/MyApp.ipa" ) ``` Then run your lane: ```bash fastlane upload ``` ## Options The `upload_to_testingbot` action accepts the following parameters. Each one also reads from the matching environment variable when it is not passed explicitly. | Parameter | Environment variable | Required | Description | | --- | --- | --- | --- | | `testingbot_key` | `TESTINGBOT_KEY` | Yes | Your TestingBot API key. | | `testingbot_secret` | `TESTINGBOT_SECRET` | Yes | Your TestingBot API secret. | | `file_path` | `TESTINGBOT_FILE_PATH` | No | Path to the local app build. Auto-resolves from the previous `gym` or `gradle` step when omitted. | | `remote_url` | `TESTINGBOT_REMOTE_URL` | No | A public URL for TestingBot to fetch the build from, instead of uploading a local file. | | `app_url` | `TESTINGBOT_REPLACE_APP_URL` | No | An existing `tb://` app identifier to replace in place, so the app keeps a stable URL across builds. | To keep a stable app identifier across builds, pass the existing `tb://` URL with `app_url`: ```ruby upload_to_testingbot( file_path: "./MyApp.ipa", app_url: "tb://YOUR_APP_KEY" ) ``` ## Output values After a successful upload the action returns a `tb://` identifier. You can reference this app in your Appium capabilities (set `app` to the `tb://` URL) or open it in [App Live](https://testingbot.com/members/apps/devices/new) for manual testing. The value is available through the lane context and environment for use later in the same lane: ```ruby lane :build_test do gradle(task: "assembleDebug") app_url = upload_to_testingbot UI.message("Uploaded app: #{app_url}") # also available via: lane_context[SharedValues::TESTINGBOT_APP_URL] lane_context[SharedValues::TESTINGBOT_APP_KEY] lane_context[SharedValues::TESTINGBOT_STORAGE_RESPONSE] ENV["TESTINGBOT_APP_URL"] ENV["TESTINGBOT_APP_KEY"] end ``` ## Notes & limits - Supported formats: `.apk` and `.aab` (Android), `.ipa` (iOS real devices) and `.zip` (iOS simulator). - Uploaded apps are stored in your [TestingBot Storage](https://testingbot.com/support/api/storage#upload) and shared with your team, so any team member can run automated or manual tests against them. - The plugin is open source and MIT licensed. Issues and pull requests are welcome on [GitHub](https://github.com/testingbot/testingbot-fastlane-plugin). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/firebase-app-distribution # Firebase App Distribution Integration Connect Firebase App Distribution to test your app releases on TestingBot's real device cloud without uploading builds by hand. Once connected, pick a Firebase **project** , an **app** and a specific **release** straight from the live App device picker, choose a device and start an interactive session. TestingBot installs the selected release on the device for you. The integration uses your own Google OAuth credentials (bring-your-own client), so TestingBot only ever has read access to the project, apps and releases you grant, and you can revoke it from Google at any time. ## Prerequisites Before connecting, set up an OAuth 2.0 client in Google Cloud for the Firebase project you want to test: - We recommend creating a **separate Firebase project** for App Live testing. - Make sure you have at least **Viewer** access to the target project, and add the teammates who need access at the project level in Firebase. - In the Google Cloud Console, create an **OAuth client ID** of type **Web application**. - Add the following **authorized redirect URI** to that OAuth client: `https://testingbot.com/members/integrations/firebase/callback` Then download the OAuth client's JSON config file (it contains the project ID, client ID and client secret), or copy those three values, you will need them in the next step. Because TestingBot uses _your own_ OAuth client, the OAuth consent screen for that client must be published, or your Google account must be added as a **test user** on it. Otherwise Google blocks the sign-in and the connection fails. ## Connect Firebase Go to [Members \> Integrations](https://testingbot.com/members/integrations) and click **Firebase App Distribution** , or open the [live App device picker](https://testingbot.com/members/apps/devices/new), choose **Firebase App Distribution** as the source and click **Connect**. ![Integrate with Firebase: upload a config file or connect with credentials](https://testingbot.com/assets/support/integrations/firebase/firebase1-a45e56a808a63bfc3fa961658dc623001f9e7468f58f5a4d27b0e796a732c603.png) You can connect in two ways: - **Upload a config file** — select the OAuth client JSON file you downloaded from Google Cloud. TestingBot reads the project ID, client ID and secret from it and fills in the form for you. - **Connect with credentials** — enter the `Project ID`, `Client ID` and `Secret key` manually. ![Connect your Firebase account using Project ID, Client ID and Secret key](https://testingbot.com/assets/support/integrations/firebase/firebase2-a014e7d3cbe0f7c2700784005230e0e1378f2558bee358d87694789df8a96c0f.png) Click **Sign in with Google**. You will be redirected to Google to choose your account and grant access. TestingBot only requests read access to your project, its apps and their releases. When you approve, you are returned to TestingBot and the connection is active. Connecting and disconnecting is an account-admin action. Regular team members can use a project that an admin has shared with the team, but cannot connect or remove the integration. ## Test a release Open the [live App device picker](https://testingbot.com/members/apps/devices/new) and choose **Firebase App Distribution** as the source. ![Select a Firebase project, app and release in the device picker](https://testingbot.com/assets/support/integrations/firebase/firebase3-99bab204b29c6dcd9d5a1b897f17ab9ac25bad90e0483ddf5c066ae8eae72e8b.png) 1. Select a **Project** from the dropdown. 2. Select an **App** (Android or iOS) in that project. 3. Select a **Release** , the dropdown shows the version and build number of each release. 4. Pick a compatible device on the right, incompatible devices (for example iOS devices for an Android build) are dimmed automatically, then click **Start session**. TestingBot fetches the selected release straight from Firebase and installs it on the device, then opens an interactive live session. There is no separate upload step. ## Manage projects You can add more than one Firebase project and decide which ones your team can use. Manage them from the [Firebase integration page](https://testingbot.com/members/integrations/firebase) or with the three-dot menu next to the project dropdown in the picker. ![Manage Firebase projects: add, share, refresh and delete](https://testingbot.com/assets/support/integrations/firebase/firebase4-c045e1f2a0276490e26689ef29c8d58c25daba0a9cc1495dd80f22f045999d4a.png) - **Add project** — add another Firebase project from the connected account to the picker. - **Share with team** — make a project visible to your whole team. Members see shared projects read-only and can launch sessions from them; only admins can add, share or delete projects. - **Refresh** — re-fetch the project's details from Firebase. - **Delete** — remove a project from the picker. This affects every team member it was shared with. ## Notes & limits - Supported formats are `.apk` (Android) and `.ipa` (iOS), matching what Firebase App Distribution stores. - TestingBot makes read-only calls to Firebase to list the project, its apps and their releases. It never modifies your Firebase project. - The release is fetched fresh from Firebase each time you start a session, so you always test the build that is currently in Firebase. - Your credentials and tokens are stored encrypted. If Google later expires or revokes the access, TestingBot prompts you to reconnect, these tokens are managed by Google, not by TestingBot. ## Disconnect To disconnect Firebase, go to the [Firebase integration page](https://testingbot.com/members/integrations/firebase) and click **Disconnect**. This removes the saved connection and projects for your whole team. You can reconnect at any time. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/github-issue-tracker # GitHub Integration Enable the GitHub integration to report bugs straight from your tests: create a new GitHub issue or update an existing one, with test metadata, screenshots and a link back to the test included automatically. A **Report a bug** button appears on every test detail page, and during a Live Testing session you can file an annotated screenshot as a GitHub issue. ## Connect GitHub Go to [Members \> Integrations](https://testingbot.com/members/integrations) and click **GitHub** , then click **Connect with GitHub**. ![Connect GitHub to TestingBot](https://testingbot.com/assets/support/integrations/github/github1-9fff2c880930f5cd6c898ee2559f82484ed806a1490d10b2ed435fc6de2b6a0e.png) You will be redirected to GitHub to authorize TestingBot. We request repository access so we can create issues on your behalf, plus read access to your organizations so we can list their repositories. We never see your password, and you can revoke access at any time from your GitHub settings. Once connected, the page shows the GitHub account TestingBot is connected as. ![GitHub connected to TestingBot](https://testingbot.com/assets/support/integrations/github/github5-16412c06c62285870d9b8de3dae367bc0e16c470b426bfa12d61eb0c2aa9316d.png) Are organization repositories not showing up after connecting? Organizations with OAuth App access restrictions require an organization admin to approve the TestingBot app in the GitHub organization settings before its repositories become available. ## Report a bug Open any test from the Test Detail page. In the header you will find a **Report a bug** button. Once one or more issues are linked to the test, the same button shows the number of linked issues instead. ![Report a bug button on the test detail page](https://testingbot.com/assets/support/integrations/github/github2-24372d0c94647d17992c856a20721b63b816e9832a55161b25b4492bfec81a3e.png) Clicking the button opens a dialog. Pick the `Organization` and `Repository` you want to file the issue in, then fill in the `Title` and `Description`. The description is pre-filled with the test metadata, and both fields support GitHub Markdown. You can optionally choose an `Assignee` from the repository's collaborators, and open **More fields** to apply one or more of the repository's `Labels`. ![Create a GitHub issue from a test](https://testingbot.com/assets/support/integrations/github/github3-feda77fac3cce54e3ad19d57b673fac65b570809b64f350451b5003b058beda3.png) Click **Create issue**. The issue is created in your repository and TestingBot shows a link to view it on GitHub. The issue is also linked to the test, so teammates opening the same test can see which issues were already reported. ## Update an existing issue Already have an issue for this bug? Switch to the **Update existing issue** tab, select one of the repository's open issues from the `Issue` dropdown and write a comment. The comment is pre-filled with the test metadata and is added to the issue on GitHub. ![Update an existing GitHub issue from a test](https://testingbot.com/assets/support/integrations/github/github4-bd10bab85339f3c3de6d4a2dc208691791d3bd94fb9d6d9ad1f54a1761b27542.png) Updating an issue links it to the test as well, so the test's issue counter includes issues you commented on. ## What gets included The pre-filled description (or comment) contains a metadata block with: - The test name, linked to a shareable test page with the video, screenshots and logs - The test result (passed or failed) - The browser or device and operating system the test ran on - The error message, when the test failed with one You can edit or remove any of this before submitting. Whoever opens the issue on GitHub can follow the test link to watch the video and inspect the logs, without needing a TestingBot account. ## Live Testing During a manual Live Testing session, click **Report Bug** to take a screenshot and annotate it in the editor. Then click **Report a bug** and choose **GitHub**. The same dialog appears; the annotated screenshot is uploaded and embedded at the top of the issue or comment, together with the session metadata. ## Disconnect To disconnect GitHub, go to the [GitHub integration page](https://testingbot.com/members/integrations/github) and click **Disconnect**. Type `Disconnect` in the confirmation dialog to confirm. ![Disconnect GitHub from TestingBot](https://testingbot.com/assets/support/integrations/github/github6-16acde16f9da99b035be428bd917adcfbe7a9159065a9987e1dce15a28ec6a7d.png) Issues that were already linked to your tests stay linked and are not removed. You can reconnect at any time; reconnecting with the same or another GitHub account does not affect previously created issues. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/trello-issue-tracker # Trello Integration Enable the Trello integration to report bugs straight from your tests: create a new Trello card or comment on an existing one, with test metadata, a screenshot and a link back to the test included automatically. A **Report a bug** button appears on every test detail page, and during a Live Testing session you can file an annotated screenshot as a Trello card. ## Connect Trello Go to [Members \> Integrations](https://testingbot.com/members/integrations) and click **Trello** , then click **Connect with Trello**. You will be redirected to Trello to authorize TestingBot. Trello shows a one-click authorization screen: click **Allow** to grant TestingBot read and write access to your boards so we can list boards and lists and create cards on your behalf. We never see your password, and you can revoke access at any time from your Trello account settings. One connection is shared by the whole team. The integration is owned by the account owner, so once it is connected every team member can report bugs to Trello without authorizing again. Once connected, the page shows the Trello account TestingBot is connected as. ![TestingBot connected to Trello](https://testingbot.com/assets/support/integrations/trello/trello1-990d6d4c8f8694a910ad148909e054dd5d37a8110eaa1baba676cfc64ad592db.png) ## Report a bug Open any test from the Test Detail page. In the header you will find a **Report a bug** button. Once one or more cards are linked to the test, the same button shows the number of linked cards instead. Click it and choose **Trello** from the menu. ![Report a bug button with GitHub, Azure DevOps and Trello options](https://testingbot.com/assets/support/integrations/trello/trello2-bbab9bf1d6e0a50b5e2724b1521c809d6de306f9baa73bc21ed14e1a4681608e.png) This opens a dialog. Pick the `Board` and the `List` you want to create the card in, then fill in the card `Name` and `Description`. The description is pre-filled with the test metadata. ![Create a Trello card from a test](https://testingbot.com/assets/support/integrations/trello/trello3-10f7c55ce04c78e89840f04948cd1f790ea6f63359d14e701a0697f7e2e8697e.png) You can optionally apply one or more of the board's `Labels` and assign one or more `Members` from the board. Click **Create card**. The card is created in your list and TestingBot shows a link to view it on Trello. The test metadata and the screenshot are attached automatically. The card is also linked to the test, so teammates opening the same test can see which cards were already reported. ## Update an existing card Already have a card for this bug? Switch to the **Comment on existing card** tab, select one of the board's open cards from the `Card` dropdown and write a comment. The comment is pre-filled with the test metadata and is added to the card on Trello, and the screenshot is attached to the card. ![Comment on an existing Trello card from a test](https://testingbot.com/assets/support/integrations/trello/trello4-a816a42a348be33f5c7ddf4aae13129db22ab61615a65897756b87400405b186.png) Commenting on a card links it to the test as well, so the test's card counter includes cards you commented on. ## What gets included The pre-filled description (or comment) contains a metadata block with: - The test name, linked to a shareable public test page with the video, screenshots and logs - The test result (passed or failed) - The browser or device and operating system the test ran on - The error message, when the test failed with one The screenshot is added to the card as an attachment. You can edit or remove any of this before submitting. Whoever opens the card on Trello can follow the test link to watch the video and inspect the logs, without needing a TestingBot account. ## Live Testing During a manual Live Testing session, click **Report Bug** to take a screenshot and annotate it in the editor. Then click **Report a bug** and choose **Trello**. The same dialog appears; the annotated screenshot is uploaded and attached to the card or comment, together with the session metadata. ## Disconnect To disconnect Trello, go to the [Trello integration page](https://testingbot.com/members/integrations/trello) and click **Disconnect**. Type `Disconnect` in the confirmation dialog to confirm. ![Disconnect Trello from TestingBot](https://testingbot.com/assets/support/integrations/trello/trello5-7683d60f6dcf59f9a548034e1363b23549fd0259304cc8309dc87b2650d3bbc6.png) Cards that were already linked to your tests stay on your tests and are not removed. You can reconnect at any time; reconnecting with the same or another Trello account does not affect previously created cards. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/asana-issue-tracker # Asana Integration Enable the Asana integration to report bugs straight from your tests: create a new Asana task or comment on an existing one, with test metadata, screenshots and a link back to the test included automatically. A **Report a bug** button appears on every test detail page, and during a Live Testing session you can file an annotated screenshot as an Asana task. ## Connect Asana Go to [Members \> Integrations](https://testingbot.com/members/integrations) and click **Asana** , then click **Connect with Asana**. ![Connect Asana to TestingBot](https://testingbot.com/assets/support/integrations/asana/asana1-3dd27b459a556c9d6a0d0c038e602ec71a3b5d441174d3736dfe23196be9def4.png) You will be redirected to Asana to authorize TestingBot. We request access to create and update tasks on your behalf so we can file work items and add comments. We never see your password, and you can revoke access at any time from your Asana account settings. Once connected, the page shows the Asana account TestingBot is connected as. ## Report a bug Open any test from the Test Detail page. In the header you will find a **Report a bug** button. Once one or more tasks are linked to the test, the same button shows the number of linked tasks instead. ![Report a bug button with the Asana option](https://testingbot.com/assets/support/integrations/asana/asana2-14aa8d2a5dbd184e68eb877e7668576f60b12876fdf5e7526b52460d221ea408.png) Click it and choose **Asana** from the menu. This opens a dialog. Pick the `Workspace` and `Project` you want to file the task in, then fill in the `Title` and `Description`. The description is pre-filled with the test metadata. ![Create an Asana task from a TestingBot test](https://testingbot.com/assets/support/integrations/asana/asana3-c5ddd32b3d9dec0d09f75649fba073cbb63999d5adbf0571c82a1fe07fddb023.png) You can optionally choose an `Assignee` from the workspace's members. Click **Create task**. The task is created in your project and TestingBot shows a link to view it on Asana. The task is also linked to the test, so teammates opening the same test can see which tasks were already reported. ## Update an existing task Already have a task for this bug? Switch to the **Update existing task** tab, select one of the project's open tasks from the `Task` dropdown and write a comment. The comment is pre-filled with the test metadata and is added to the task as a new comment on Asana. Commenting on a task links it to the test as well, so the test's task counter includes tasks you commented on. ## What gets included The pre-filled description (or comment) contains a metadata block with: - The test name, linked to a shareable test page with the video, screenshots and logs - The test result (passed or failed) - The browser or device and operating system the test ran on - The error message, when the test failed with one You can edit or remove any of this before submitting. Whoever opens the task on Asana can follow the test link to watch the video and inspect the logs, without needing a TestingBot account. ## Live Testing During a manual Live Testing session, click **Report Bug** to take a screenshot and annotate it in the editor. Then click **Report a bug** and choose **Asana**. The same dialog appears; the annotated screenshot is uploaded to the task as an attachment and shown inline at the top of the task or comment, together with the session metadata. ## Disconnect To disconnect Asana, go to the [Asana integration page](https://testingbot.com/members/integrations/asana) and click **Disconnect**. Type `Disconnect` in the confirmation dialog to confirm. Tasks that were already linked to your tests stay linked and are not removed. You can reconnect at any time; reconnecting with the same or another Asana account does not affect previously created tasks. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/azure-devops-issue-tracker # Azure DevOps Integration Enable the Azure DevOps integration to report bugs straight from your tests: create a new Azure DevOps work item or update an existing one, with test metadata, screenshots and a link back to the test included automatically. A **Report a bug** button appears on every test detail page, and during a Live Testing session you can file an annotated screenshot as an Azure DevOps work item. ## Connect Azure DevOps Go to [Members \> Integrations](https://testingbot.com/members/integrations) and click **Azure DevOps** , then click **Connect with Azure DevOps**. ![Connect Azure DevOps to TestingBot](https://testingbot.com/assets/support/integrations/azure/azure1-69bc9452fbe870d86f823563247a590d2adf39dbd84253fc308bb4da80933cf1.png) You will be redirected to Microsoft to authorize TestingBot. We request work item access (`vso.work_write`) so we can create and update work items on your behalf, plus project access (`vso.project`) so we can list your organizations and projects. We never see your password, and you can revoke access at any time from your Azure DevOps profile settings. Once connected, the page shows the Azure DevOps account TestingBot is connected as. Is an organization not showing up after connecting? Some organizations block OAuth apps by default. An organization administrator needs to enable **Third-party application access via OAuth** in **Organization Settings \> Security \> Policies** before TestingBot can access that organization. ## Report a bug Open any test from the Test Detail page. In the header you will find a **Report a bug** button. Once one or more work items are linked to the test, the same button shows the number of linked work items instead. ![Report a bug button with GitHub and Azure DevOps options](https://testingbot.com/assets/support/integrations/azure/azure2-5886a72dcda4df13c6a07ba5b926a192790b84d62e869c5c144114b02975c727.png) Click it and choose **Azure DevOps** from the menu. This opens a dialog. Pick the `Organization` and `Project` you want to file the work item in, choose a `Work Item Type` (for example Bug, Task or User Story, depending on your project's process template), then fill in the `Title` and `Description`. The description is pre-filled with the test metadata. You can optionally choose an `Assignee` from the project's team members. Click **Create work item**. The work item is created in your project and TestingBot shows a link to view it on Azure DevOps. The work item is also linked to the test, so teammates opening the same test can see which work items were already reported. ## Update an existing work item Already have a work item for this bug? Switch to the **Update existing work item** tab, select one of the project's recent work items from the `Work Item` dropdown and write a comment. The comment is pre-filled with the test metadata and is added to the work item's discussion on Azure DevOps. Updating a work item links it to the test as well, so the test's work item counter includes work items you commented on. ## What gets included The pre-filled description (or comment) contains a metadata block with: - The test name, linked to a shareable test page with the video, screenshots and logs - The test result (passed or failed) - The browser or device and operating system the test ran on - The error message, when the test failed with one You can edit or remove any of this before submitting. Whoever opens the work item on Azure DevOps can follow the test link to watch the video and inspect the logs, without needing a TestingBot account. ## Live Testing During a manual Live Testing session, click **Report Bug** to take a screenshot and annotate it in the editor. Then click **Report a bug** and choose **Azure DevOps**. The same dialog appears; the annotated screenshot is uploaded and attached at the top of the work item or comment, together with the session metadata. ## Disconnect To disconnect Azure DevOps, go to the [Azure DevOps integration page](https://testingbot.com/members/integrations/azure) and click **Disconnect**. Type `Disconnect` in the confirmation dialog to confirm. Work items that were already linked to your tests stay linked and are not removed. You can reconnect at any time; reconnecting with the same or another Azure DevOps account does not affect previously created work items. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/jira # Jira Integration Enable the Jira integration to create Jira issues directly from TestingBot, with test metadata, logs, video and screenshots attached. You can connect in two ways: link your Jira Cloud account in one click with Atlassian OAuth, or enter an API token for Jira Server and Data Center. ## Connect with Atlassian This is the quickest way to connect a Jira Cloud site. Go to [Members \> Integrations](https://testingbot.com/members/integrations) and click **Jira** , then click **Connect with Atlassian**. ![Connect Jira to TestingBot with Atlassian](https://testingbot.com/assets/support/integrations/jira/jira1-b31411832a653cca0d449af26550e60a739745bb5c23aed6476bed2f7b0f8955.png) You will be redirected to Atlassian to approve TestingBot. We request read and write access to Jira work items, so we can read your projects and issue types and create issues, plus offline access so we can keep creating issues without asking you to sign in again. We never see your password, and you can revoke access at any time from your Atlassian account settings. TestingBot does not store your Atlassian account details. Once connected, the page shows the connected site and a **Test connection** button you can use to confirm everything works. ![Jira connected to TestingBot](https://testingbot.com/assets/support/integrations/jira/jira2-9b1a9aa403d21bc910fcf8e93f075de43ef507e56cee410152bd2f9964d23857.png) ## Choose a Jira site If your Atlassian login has access to more than one Jira site, TestingBot asks which site it should create issues in right after you approve the connection. Pick the site and click **Use this site**. If you only have one site, this step is skipped. You can switch sites later by disconnecting and connecting again. ## Use an API token Prefer a token, or running Jira Server or Data Center, which does not support the Atlassian OAuth flow? On the Jira integration page, open **Use an API token instead** and fill in the `Jira email` you sign in with, an `API token`, and the `Atlassian host` of your Jira instance (for example `https://yourcompany.atlassian.net/`). ![Connect Jira with an API token](https://testingbot.com/assets/support/integrations/jira/jira3-0139409773eb0a74d3cbbf6ed7fb3b085b4f8f47e08066c35240820221fe9fe3.png) Jira Cloud no longer accepts account passwords for REST API access. Generate an API token from [Atlassian, Security, API tokens](https://id.atlassian.com/manage-profile/security/api-tokens) and use that instead. Jira Server and Data Center users can supply a Personal Access Token (PAT) in the same field. We store these credentials encrypted and only use them to create issues on your behalf. ## Report a bug Once Jira is connected, you can create an issue from any automated test. Open the test from the Test Detail page, click the **Report a bug** button in the header and choose **Jira**. ![Report a bug menu with the Jira option](https://testingbot.com/assets/support/integrations/jira/jira4-0c71ecdd989ec22732b65259f564665c2ed8845b73277a5fcee2f8136545aa21.png) A form appears where you pick the `Project` and `Issue type`, then fill in the `Title` and `Description`. The issue type list updates to match the project you select. The description is pre-filled with the test metadata, including the test URL, environment, name and result. ![Create a Jira issue from a test](https://testingbot.com/assets/support/integrations/jira/jira5-ffb7550be5de2bf6fdfa66a8eb191e105b743cedc1fd0579690caafd82f1f676.png) You can also choose which test artifacts to attach: the console and driver logs, the network log (HAR) and the video recording. Attachments are added to the issue in the background. Click **Create issue**. The Jira ticket is created in your Jira instance and TestingBot shows a link to view it in Jira. ## What gets included The pre-filled description contains a metadata block with: - The test URL, which opens a shareable test page with the video, screenshots and logs - The environment the test ran on, the browser or device and the operating system - The test name - The test result (passed or failed) You can edit or remove any of this before submitting. Whoever opens the issue in Jira can follow the test link to watch the video and inspect the logs, without needing a TestingBot account. When you tick the artifact checkboxes, the logs, the network log and the video are uploaded to the issue as attachments. ## Live Testing During a manual Live Testing session, click **Report Bug** to take a screenshot and annotate it in the editor. Then click **Report a bug** and choose **Jira**. The same issue form appears, and the annotated screenshot is attached to the issue you create, together with the session metadata. ## Disconnect To disconnect Jira, go to the [Jira integration page](https://testingbot.com/members/integrations/jira) and click **Remove Jira integration** , then confirm. ![Disconnect Jira from TestingBot](https://testingbot.com/assets/support/integrations/jira/jira6-43bb06903febdfde6d2a8d320900818c908316993d353cdacda3263db21689e3.png) Issues that were already created in Jira are not affected. You can reconnect at any time, with the same or another Jira account, and with Atlassian OAuth or an API token. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/katalon-studio # Katalon Studio Plugin TestingBot has a [Katalon Studio Plugin](https://store.katalon.com/product/155/TestingBot-Integration) available in the Katalon Store. ## Install Plugin Navigate to the [plugin page](https://store.katalon.com/product/155/TestingBot-Integration), and click **Install Plugin**. The plugin will be added to your Katalon Studio. ## Configure Plugin Once installed, go to **Project \> Settings**. Under the Plugins section, you will see the TestingBot Integration plugin. Fill in the TestingBot `KEY` and `SECRET`, together with the desired capabilities you want to test. ![Katalon Studio + TestingBot](https://testingbot.com/assets/support/katalon/katalon1-e886ffe2cff674d78c9cebd21e80e7534cb825c4e1adc2d3a95e9815c684abe4.png) ## Run Test To run the test on TestingBot: - Click the Run button `(Command + Shift + A)` - Select Custom capabilities - Select the TestingBot profile ![Katalon Studio + TestingBot](https://testingbot.com/assets/support/katalon/katalon2-1fdf4a496b469d06c2e805c89330147f27184eb426846b64a9b74da4f8bc2c5f.png) ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/qmetry # QMetry QMetry Automation Studio offers multi-language scripting to automated testing. The product provides partial codeless automation, targeted to manual users or beginners. The framework offers a test recorder which allows users to record test cases and generate code. This guide documents how to use QMetry with TestingBot to run automated tests on various platforms. ## Configuration To get started with running a cloud test using QMetry, you'll need to specify some settings: 1. Add `driver.name=remotedrivername` in either **application.properties** , **env.properties** or **testrun\_config.xml** Where `remotedrivername` is either **chromeRemoteDriver** , **firefoxRemoteDriver** or another valid remote drivername. 2. Set the `remote.server` to use the TestingBot Hub: 3. Specify the capabilities for your test. You'll need to specify this in the same file. 4. Update the `driver.name` parameter in `web.config` 5. You can now run the test from the command line. The test will run on the TestingBot browser grid. ## TestingBot Tunnel You can use [TestingBot Tunnel](https://testingbot.com/support/tunnel) to test websites that are not publicly available. For example a webpage on your staging server, or local computer. To get started, please see the steps below: - Download and run [TestingBot Tunnel](https://testingbot.com/support/tunnel) with a specific tunnel identifier that you chose: - Now you will need to specify the same `tunnel-identifier` in the capabilities, to let TestingBot know it should use the tunnel you just created: ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/ranorex # Integrate TestingBot with Ranorex Ranorex is a GUI test framework, designed to execute tests. Using Ranorex Studio, you can easily manage and record tests, in your local browser. Without writing any code, you can create advanced Selenium WebDriver test cases. This guide will help you in setting up a connection between Ranorex Studio and TestingBot, allowing you to run tests created with Ranorex on TestingBot's remote browser grid. ## Getting started with Ranorex 1. Start Ranorex Studio and click the **Web** tab in the wizard to create a new Web test. 2. Provide the Solution name and Directory Location to store this new Project, then click **Continue**. 3. Specify the URL of the website you'd like to test and choose a browser in the next screen. Next, click **Continue**. 4. Select **Do not use whitelisting** as recording behavior. 5. Select **Finish** to complete this solution. ## Setting up TestingBot Endpoint in Ranorex Studio Before you can run a Ranorex Studio test on TestingBot, we'll first have to set up TestingBot as an endpoint. To do this, please follow the steps below. 1. To run the test on TestingBot's remote browser grid, add the TestingBot endpoint by selecting **View** in the menubar, then click **Endpoints**. Or type `Ctrl+Alt+P`. 2. In the dialog that appears, click the **Add endpoint** button. 3. Select **WebDriver** as the Endpoint type. As the Endpoint name, you can fill in TestingBot. The address should be `https://hub.testingbot.com/wd/hub`. Finally, click the **Add endpoint** button. 4. You have now successfully added TestingBot as an Endpoint. Click the icon next to TestingBot's endpoint to activate it (making it green). ## Executing Ranorex tests on TestingBot 1. You will need to add the necessary capabilities to the Endpoint's **Capabilities JSON** section. To do this, right click the TestingBot connector and click **View details**. 2. Add the following capabilities in **Capabilities JSON section** and add a name in the Name field. Finally, click the **Save** button. 3. Now mark the configuration you just created as the **Active configuration**. 4. Before we can run tests, we will need to increase the default timeout value of 10 seconds. To do this, open the settings by clicking the **Open Settings** button. 5. With the Settings window open, click the **Plugins** tab and scroll to the **WebDriver** section, where you will see a **WebDriver command timeout** value of 10. 6. To run a test, make sure you've selected the TestingBot endpoint. 7. You can now run your Ranorex test on TestingBot. Click the **Run** button. Once the test has finished, you will see it appear in the TestingBot dashboard with a video, logs and other metadata. ## Testing privately hosted websites with Ranorex and TestingBot TestingBot allows you to run automated tests on internal development environments such as staging URLs or localhost. To get started, please make sure you've downloaded [TestingBot Tunnel](https://testingbot.com/support/tunnel). You can start the tunnel through the commandline: ```bash java -jar testingbot-tunnel.jar key secret --tunnel-identifier ranorexTunnel ``` Next, you will need to edit the **Capabilities JSON** configuration in the custom TestingBot endpoint you've created and add a `tunnelIdentifier` in the `tb:options` JSON object. ```json { "tb:options": { "key": "....", "secret": "...", "tunnelIdentifier": "ranorexTunnel" } } ``` ## Passing custom capabilities with Ranorex Studio TestingBot provides various [custom capabilities](https://testingbot.com/support/web-automate/selenium/test-options) to configure test runs, such as changing the `browserName`, `browserVersion`, `platformName` and more. To include these with your test, make sure to edit the **Capabilities JSON** configuration in the custom TestingBot endpoint you've created. ```json { "browserName": "chrome", "browserVersion": "latest", "platformName": "WIN11", "tb:options": { "key": "....", "secret": "...", "screen-resolution": "1280x1024", "name": "Ranorex Test" } } ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/slack # Slack Integration Add the Slack TestingBot app to your workspace to integrate TestingBot with your Slack Workspace. ## Features Once you've added the [TestingBot Slack App](https://slack.com/apps/A9L319MDY-testingbot) to your Slack workspace, these features will become available: - Our Slack Bot can **post when a test finishes, succeeds or fails.** - **Slash commands** : see Test Analytics, Builds and other test-metadata straight from inside Slack. - **Unfurl URLs** : paste a TestingBot test URL in your Slack channel and our bot will add extra meta-data to the post. - **Share test-results** with other Slack members. - **Chat** with our Slack TestingBot App. ## Getting Started To get started, please log in and go to the [Slack Integration page](https://testingbot.com/members/integrations/slack). You will see a button "Add to Slack" which will redirect to the slack.com website. There, you will be asked to add the TestingBot app to your workspace. Pick which channel you'd like our TestingBot app to join. ![Setup Slack Integration](https://testingbot.com/assets/support/slack/setup-4e4f827f057250cb4e0fd5fe6552c7ed72c7f7ebe62c943d96ff758e42d73161.png) ## Test Events Once you've added the TestingBot app to your Slack workspace, you can specify if you'd like to be notified when a test on TestingBot succeeds or fails. For example: as soon as a test fails, our bot will post to the Slack channel you specified. The notification will include test meta-data and a direct link to the test where you can see a video recording, logs and screenshots of the test. To specify for which events you'd like to be notified, please go to the [Slack Settings page](https://testingbot.com/members/integrations/slack) in our member area. There you'll be able to specify if and when we should send Test-Events to your Slack channel. - Send a message to your Slack channel when a test finishes - Send a message to your Slack channel, but only when a test succeeds - Send a message to your Slack channel, but only when a test fails You can optionally set a condition to only send out these Slack events for a [specific group](https://testingbot.com/support/web-automate/selenium/test-options#grouping). ## Slack Commands The TestingBot app will add the following commands to your Slack workspace: - ```bash /tests ``` Retrieves an overview of your most recent tests. For each test you can see if it passed or failed, how long it took and on which platform it ran. You can specify a specific day to only show tests for that day. These are the possible arguments you can specify: - today - yesterday - week (last 7 days) - month (last 31 days) - `YYYY-MM-DD` format - ```bash /test ``` Shows details for a single test. Details include test duration, success state, a screenshot and which platform the test was run on. - ```bash /builds ``` Shows an overview of the most recent builds. How many tests succeeded in the build, how many failed. A link is included to directly go the build overview page. - ```bash /build ``` Shows details for a single build. Specify the build-identifier. For example: ```bash /build my-build-identifier ``` ## Unfurl URLs When you or your colleagues on Slack post a TestingBot URL, and the URL links to a test result, then the TestingBot Slack app will automatically add test meta-data to the link. Test meta-data includes: - Which browser/platform the test ran on - Whether the test succeeded or failed - A screenshot of the test - How long the test took to run ## Private Channels To use our Slack bot in a private channel, please add the app manually to your private channel. Click the name of your private channel, then choose **Add an app**. Once you've added the app, our Slackbot will be able to communicate in your private channel. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/tricentis-tosca # TestingBot and Tricentis Tosca [Tricentis Tosca](https://www.tricentis.com/products/automate-continuous-testing-tosca) is an industry-leading test creation tool, that allows for testing of websites and mobile apps. By connecting to TestingBot from within Tosca Commander, you will be able to run your tests in parallel on remote browsers and devices. To get ready, please make sure you have downloaded and installed Tosca. Next, you'll need a TestingBot account to pass your TestingBot key and secret credentials to Tosca. ## Running tests on TestingBot With Tosca Commander open, make sure you have created or recorded some web test cases. For a specific test case, go to the **Test configuration** tab, and make sure these test configuration parameters are listed: - **Browser** : set this to Chrome, Firefox, Edge or another browser - **OperatingSystem** : for example `WIN10`, `WIN11` or [other platforms](https://testingbot.com/support/web-automate/browsers) - **HubAddress** : set this to `https://TB_KEY:TB_SECRET@hub.testingbot.com/wd/hub`, where you replace `TB_KEY` with your actual TestingBot key and `TB_SECRET` with your actual TestingBot secret, obtained from the member area ![Tosca Configuration](https://testingbot.com/assets/support/integrations/tosca/config-aaf0eb391f94110e69124271012085d2173eb54861b887e47349a8e6500793f1.webp) ### Set Desired Capabilities With the [Set Desired Capabilities](https://documentation.tricentis.com/tosca/2410/en/content/engines_3.0/mobile/tbox_mobileweb_set_appium_capabilities.htm) module you can specify multiple capabilities to target and customize tests on TestingBot. Make sure to use this module at the start of your test(s). The **Set Desired Capabilities module** is part of the Standard subset. You can find it under `Standard modules->TBox XEngines->Mobile`. ![Tosca Set Desired Capabilities](https://testingbot.com/assets/support/integrations/tosca/capabilities-module-88cb4211690cbbe02bed828785ffe38ff103b3c772e50c41038c1935ae8b231c.webp) You will need to specify the `AppiumServer` option, specifying the TestingBot Grid endpoint `https://hub.testingbot.com/wd/hub`. ![Tosca AppiumServer](https://testingbot.com/assets/support/integrations/tosca/appium-server-fb43069ff59a569cef71aefa870a7db977e9069d64713673e1e09b0b0f9a1eeb.webp) You can specify the `Fibermode = true` option to improve the performance of remote WebDriver and Tosca. It will decrease the amount of communication that is required between Tosca and TestingBot, improving overall performance. ### TestingBot Storage You can run mobile app tests with Tosca on TestingBot's mobile devices, by specifying for example a [TestingBot Storage URL](https://testingbot.com/support/api/storage#upload). You can add a `.apk` or `.ipa` extensions to the `tb://` URLs, to make sure Tosca accepts these URLs. ![Tosca TestingBot Storage](https://testingbot.com/assets/support/integrations/tosca/tb-url-865a51ae63ffe7f82cdf889dd3d8a860cf4f4611b15cc16ccb0b3703599f03cf.webp) ## Scanning a Mobile App Tosca Commander provides a way to scan a mobile iOS or Android app. You can use this on a remote iOS or Android device running on TestingBot. 1. In Tosca Commander, on the **Modules** tab, click **Scan** and then click **Mobile**. 2. In the **Add connection** input, enter `https://hub.testingbot.com/wd/hub` as Appium Server Address. Choose **Cloud (Appium)** as type. 3. In the **Add device** input field, enter the information for the TestingBot device you want to use. For example, **Pixel 8**. 4. In the **Edit native/hybrid application input** , enter the URL to the app you want to test, or supply the [TestingBot Storage](https://testingbot.com/support/api/storage#upload) URI, then click **Save**. 5. Now we'll need to add the TestingBot key and secret, to authenticate with the TestingBot grid. Click **Advanced Configuration** , then **Add Capabilities Set** and click the **Add capability** button to add both **key** and **secret** capabilities, with their values being your actual TestingBot key and secret, obtained from the member area. 6. In the **Establishing connection** input, click **Scan**. After a while, you will see a success message. 7. You now have access to a live view of the device running on TestingBot. You can interact with the device by using the buttons. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/zebrunner # Zebrunner Zebrunner offers test management with smart analysis of test results. Run and view tests from the Zebrunner UI on TestingBot through the [Zebrunner and TestingBot integration](https://zebrunner.com/documentation/integrations/testingbot). The Zebrunner - TestingBot integration provides Zebrunner users the following features: - Launch TestingBot automated tests - View test results, including screenshots, logs and a video from the Zebrunner UI - Access to TestingBot credentials ## Configuration To configure this integration, please perform the following steps: 1. Log in to your [TestingBot member dashboard](https://testingbot.com/members) and copy your **TestingBot Key** and **TestingBot Secret**. 2. Log in to Zebrunner, select a project and click **Integrations**. 3. Select the TestingBot integration, and enter the following details: 4. Click **Save** to save the configuration. It is now possible to launch tests on the TestingBot browser grid. ## Run Test To run a test, go to the project and open the **Launcher** view where you can select the TestingBot integration. You can choose a browser/version/OS combination with the dropdown menus. ![Zebrunner and TestingBot](https://testingbot.com/assets/support/zebrunner/zebrunner-demo-6b9a40a242a45528d8f8d167e5eb93561998b62e515d5b4b8b00edbb30cfa5c6.png) ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/webhooks # Webhooks Integration Webhooks let TestingBot push test results to your own systems the moment a test completes. Rather than polling for updates through the [TestingBot API](https://testingbot.com/support/api), you receive an HTTP request with the test details as soon as they become available. Use webhooks to notify chat tools such as Slack, Microsoft Teams or Discord, open incidents in PagerDuty, OpsGenie or incident.io, create Jira tickets, or feed test data into your own dashboards and reporting pipelines. ## How it works 1. A test finishes on TestingBot and matches the triggers you configured for the webhook. 2. TestingBot renders your payload: either the default JSON payload, or a custom body with [built-in variables](https://testingbot.com/support/integrations/webhooks/built-in-variables) filled in with the test details. 3. TestingBot sends an HTTP request to your URL, with the authorization, headers and query parameters you configured. ## Explore the documentation - [**Quick Start**](https://testingbot.com/support/integrations/webhooks/quick-start): create and test your first webhook in a few minutes. - [**Core Concepts**](https://testingbot.com/support/integrations/webhooks/core-concepts): triggers, payload modes, the default payload reference, delivery behavior and limits. - [**Built-in Variables**](https://testingbot.com/support/integrations/webhooks/built-in-variables): the full reference of variables you can use in custom payloads, URLs, headers and parameters. - [**Integrations**](https://testingbot.com/support/integrations/webhooks/integrations): connect Slack, Teams, Discord, PagerDuty, OpsGenie, incident.io or Jira with a ready-made template. - [**Slack**](https://testingbot.com/support/integrations/webhooks/integrations/slack): post a test summary to a Slack channel. - [**Microsoft Teams**](https://testingbot.com/support/integrations/webhooks/integrations/microsoft-teams): send a card to a Teams channel. - [**Discord**](https://testingbot.com/support/integrations/webhooks/integrations/discord): post an embed to a Discord channel. - [**PagerDuty**](https://testingbot.com/support/integrations/webhooks/integrations/pagerduty): open an incident when a test fails. - [**OpsGenie**](https://testingbot.com/support/integrations/webhooks/integrations/opsgenie): create an alert for failed tests. - [**incident.io**](https://testingbot.com/support/integrations/webhooks/integrations/incident-io): fire an alert for failed tests. - [**Jira**](https://testingbot.com/support/integrations/webhooks/integrations/jira): create a Jira issue with the test details. ## At a glance - Up to **5 webhooks per team** , managed in the [members area](https://testingbot.com/members/integrations/webhooks). - Requests can use the **POST** , **PUT** or **PATCH** method. - Optional **Basic Auth** or **Bearer token** authorization. - Custom **headers** and **query parameters** , with variable support. - A **Test Webhook** button that sends a sample request and shows the live response. - Every delivery is **signed with HMAC-SHA256** so your receiver can [verify it comes from TestingBot](https://testingbot.com/support/integrations/webhooks/core-concepts#verify). Ready to start? Follow the [Quick Start](https://testingbot.com/support/integrations/webhooks/quick-start), or read the [Core Concepts](https://testingbot.com/support/integrations/webhooks/core-concepts) to understand triggers, payloads and delivery in detail. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/webhooks/quick-start # Webhooks Quick Start This guide takes you from zero to a working webhook in a few minutes. You will create a webhook in the TestingBot member area, choose when it fires, pick a payload and verify it with a test request. ## Prerequisites - A TestingBot account. Webhooks are configured in the [members area](https://testingbot.com/members/integrations/webhooks). - An HTTP endpoint to receive the requests. If you do not have one yet, you can use one of the [ready-made templates](https://testingbot.com/support/integrations/webhooks/integrations) for services such as Slack, Microsoft Teams, Discord, PagerDuty, OpsGenie or Jira. ## Create your first webhook 1. Navigate to the [Integrations page](https://testingbot.com/members/integrations) in the member area. 2. Click the **Webhooks** integration. 3. Click the **Create webhook** button. You can create up to 5 webhooks per team. 4. Give the webhook a **name** so you can recognize it later in the overview. 5. Choose when the webhook should fire: **failures only** or all tests, which **test type** to include (all tests, automated browser tests or real device tests), and optionally a **name filter** such as `regression_*` to only match tests or builds with a specific name pattern. See [when webhooks fire](https://testingbot.com/support/integrations/webhooks/core-concepts#events) for details. 6. Pick the payload: keep the **Default** payload, switch to **Custom** and write your own JSON body with [built-in variables](https://testingbot.com/support/integrations/webhooks/built-in-variables), or click one of the **Start from a template** tiles to prefill everything for a popular service. 7. Optionally configure **authorization** (Basic Auth or a Bearer token). In **Custom** payload mode you can also add extra **headers** and **query parameters** in the corresponding tabs of the form. 8. Click **Test Webhook** and check the response panel to verify your endpoint accepts the request. 9. Click **Create webhook** to complete the configuration. Your webhook now appears in the overview page, where you can edit or delete it. Once these steps are completed, you will start receiving webhook requests on the URL you specified. Requests may arrive with a short delay after a test completes. ## Send a test While creating or editing a webhook, click the **Test Webhook** button to send a sample request to your URL right away. The test uses sample values for all variables, so you can verify your endpoint and payload without running an actual test. The **Response** panel below the button shows the HTTP status of your endpoint, how long the request took, the response headers and body, and the exact payload that was sent. Use it to confirm your endpoint returns a 2xx status before saving the webhook. ## Next steps - Read the [Core Concepts](https://testingbot.com/support/integrations/webhooks/core-concepts) for the default payload reference, delivery behavior and limits. - Browse the [Built-in Variables](https://testingbot.com/support/integrations/webhooks/built-in-variables) to compose custom payloads. - Pick a ready-made setup from the [Integrations](https://testingbot.com/support/integrations/webhooks/integrations) overview. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/webhooks/core-concepts # Webhooks Core Concepts This page explains how TestingBot webhooks behave: when they fire, what they send, how the request is authorized and what delivery guarantees apply. If you have not created a webhook yet, start with the [Quick Start](https://testingbot.com/support/integrations/webhooks/quick-start). ## When webhooks fire A webhook fires after each completed test that matches its trigger settings. You configure three filters when creating or editing a webhook at [/members/integrations/webhooks](https://testingbot.com/members/integrations/webhooks): - **Result filter** : fire on **failures only** , or on all tests regardless of their result. - **Test type** : include all tests, only **automated browser tests** or only **real device tests**. - **Name filter** : an optional wildcard pattern such as `regression_*`. The webhook only fires when the pattern matches the test name or the build name. Requests may arrive with a short delay after a test completes. ## Default and custom payloads Each webhook sends one of two payload types, selected with the **Default | Custom** toggle in the webhook form: - **Default** : a fixed JSON document with all the test details, documented in the [reference below](https://testingbot.com#default-payload). This is the easiest option when your own endpoint consumes the request. - **Custom** : a JSON body that you write yourself, using [built-in variables](https://testingbot.com/support/integrations/webhooks/built-in-variables) such as `{{TEST_NAME}}` and `{{TEST_STATUS}}`. Use this to match the format an external service expects, or pick one of the [templates](https://testingbot.com/support/integrations/webhooks/integrations) to prefill a custom payload for a popular service. ## Default payload reference When you do not configure a custom payload, your webhook URL receives a POST request in JSON format with the following payload: **Changed August 2026:** `browser_name` is now a stable identifier without the version (for example `googlechrome` rather than `Chrome 150`). The human-readable label moved to `browser_display_name`. If a custom payload builds a label from `{{BROWSER_NAME}} {{BROWSER_VERSION}}`, switch it to `{{BROWSER_DISPLAY_NAME}}`. | Data Field | Format | Description | | --- | --- | --- | | `id` | STRING | The unique id of the test. Usually this will be a WebDriver SessionId, or Playwright/Puppeteer unique id. | | `creation_time` | DATETIME | The date-time value, in `YYYY-MM-DDTHH:mm:ssZ` format, at which the test launched. | | `completion_time` | DATETIME | The date-time value, in `YYYY-MM-DDTHH:mm:ssZ` format, at which the test was completed. | | `user_id` | INTEGER | The id of the user who launched the test. | | `team_id` | INTEGER | The id of the team for which this test was run. | | `status` | ENUM | The status of the test, which can be one of the following values: - `PASSED` - `FAILED` - `UNKNOWN` | | `name` | STRING | The name of the test, if it was passed through the [capabilities](https://testingbot.com/support/web-automate/selenium/test-options#name) or [set by API](https://testingbot.com/support/api/tests#updatetest). | | `browser_name` | STRING | The browser identifier, without the version (for example `googlechrome`, `iexplore`, `firefox`). Stable across versions, so this is the field to match on. In case of a mobile app test, this might be `NULL`. | | `device_name` | STRING | The name of the device on which this test ran. Might be `NULL` when this test ran on a Desktop environment. | | `browser_version` | STRING | The version of the browser on which this test ran, as a string rather than a number, so it may carry minor or patch components (for example `150`, `11.4`). In case of a mobile app test, this might be `NULL` or the device's version. | | `browser_long_version` | STRING | The full version string when we know it (for example `121.0.6167.184`), otherwise empty. | | `browser_display_name` | STRING | A human-readable label for dashboards and chat messages (for example `Chrome 150`), with the device and OS appended on mobile. The version is already part of this string, so do not append `browser_version` to it. Presentation only: the format is not stable, so never match on it. | | `platform` | STRING | The OS on which this test ran. | | `duration_sec` | INTEGER | How long the test took to complete, the duration is in seconds. | | `build` | STRING | The build this test belongs to, if it was passed through the [capabilities](https://testingbot.com/support/web-automate/selenium/test-options#build) or [set by API](https://testingbot.com/support/api/tests#updatetest). | | `tags` | ARRAY | Array of strings that were added for this test, if it was passed through the [capabilities](https://testingbot.com/support/web-automate/selenium/test-options) or [set by API](https://testingbot.com/support/api/tests#updatetest). | | `errors` | ARRAY | Array of error objects that were reported for this test. Empty if no errors were reported. The error objects are structured like this: ```json errors = [{ message: "The error message", test_name: "The name of the sub test, if any. For example a Flow name when testing with Maestro" }] ``` | ## Authorization, headers and parameters The **Authorization** , **Headers** and **Params** tabs in the webhook form control how the HTTP request is sent. The tabs appear when the **Custom** payload mode is selected; with the **Default** payload, only the Authorization fields are shown: - **Authorization** : choose **No Auth** , **Basic Auth** (username and password) or a **Bearer token**. The credentials are added to the `Authorization` header of every request. - **Headers** : add custom HTTP headers, for example an API key header that the receiving service requires. - **Params** : add query parameters that are appended to the webhook URL. Header values, query parameter values and the URL itself support [built-in variables](https://testingbot.com/support/integrations/webhooks/built-in-variables), so you can include test details such as `{{TEST_ID}}` outside of the payload body. ## Delivery and security - Delivery is **best effort** and is **not retried** : make sure your endpoint is available and responds quickly. - Your endpoint is expected to return a **2xx** status code. - Requests time out after **90 seconds**. - **Redirects are not followed** : point the webhook directly at the final URL. - Every delivery is **signed** with an HMAC-SHA256 signature, so your endpoint can [verify that the request really comes from TestingBot](https://testingbot.com#verify). You can additionally configure [authorization](https://testingbot.com#auth) as a second factor. - Webhook URLs that resolve to **private or internal addresses are refused** ; the endpoint must be reachable on the public internet. ## Verify deliveries Each webhook has a **signing secret** (it starts with `whsec_`), shown on the webhook's edit page in the [members area](https://testingbot.com/members/integrations/webhooks), where you can also rotate it. TestingBot signs every delivery with it and adds two headers to the request: | Header | Contents | | --- | --- | | `X-TestingBot-Timestamp` | The Unix timestamp (seconds) at which the request was signed. | | `X-TestingBot-Signature` | `sha256=`: the HMAC-SHA256 of `"."`, computed with your signing secret. | To verify a delivery on your server: 1. Read the **raw request body** , before any JSON parsing or re-serialization. A single re-encoded space breaks the signature. 2. Reject the request when the timestamp is **older than 5 minutes** ; this blocks replayed deliveries. 3. Compute the HMAC-SHA256 of `"."` with your signing secret and prefix it with `sha256=`. 4. Compare it with the `X-TestingBot-Signature` header using a **constant-time comparison** , never a plain string equality. [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [C#](https://testingbot.com#) ```java import javax.crypto.Mac; import javax.crypto.spec.SecretKeySpec; import java.nio.charset.StandardCharsets; import java.security.MessageDigest; public class TestingBotWebhook { public static boolean verifySignature(String rawBody, String timestamp, String signature, String secret) throws Exception { if (Long.parseLong(timestamp) < (System.currentTimeMillis() / 1000) - 300) { return false; } Mac mac = Mac.getInstance("HmacSHA256"); mac.init(new SecretKeySpec(secret.getBytes(StandardCharsets.UTF_8), "HmacSHA256")); byte[] digest = mac.doFinal((timestamp + "." + rawBody).getBytes(StandardCharsets.UTF_8)); StringBuilder hex = new StringBuilder(); for (byte b : digest) { hex.append(String.format("%02x", b)); } String expected = "sha256=" + hex; // MessageDigest.isEqual is a constant-time comparison. return MessageDigest.isEqual( expected.getBytes(StandardCharsets.UTF_8), (signature == null ? "" : signature).getBytes(StandardCharsets.UTF_8)); } } ``` ```javascript const crypto = require('crypto'); function verifyTestingBotSignature(rawBody, timestamp, signature, secret) { if (Number(timestamp) < Math.floor(Date.now() / 1000) - 300) return false; const expected = 'sha256=' + crypto.createHmac('sha256', secret) .update(`${timestamp}.${rawBody}`) .digest('hex'); const a = Buffer.from(expected); const b = Buffer.from(String(signature || '')); return a.length === b.length && crypto.timingSafeEqual(a, b); } // Express needs the raw body, for example: // app.use(express.json({ verify: (req, res, buf) => { req.rawBody = buf.toString(); } })); ``` ```ruby require 'openssl' require 'rack/utils' def verify_testingbot_signature(raw_body, timestamp, signature, secret) return false if timestamp.to_i < Time.now.to_i - 300 expected = 'sha256=' + OpenSSL::HMAC.hexdigest('SHA256', secret, "#{timestamp}.#{raw_body}") Rack::Utils.secure_compare(expected, signature.to_s) end # Rails example: # verify_testingbot_signature( # request.raw_post, # request.headers['X-TestingBot-Timestamp'], # request.headers['X-TestingBot-Signature'], # ENV['TESTINGBOT_WEBHOOK_SECRET'] # ) ``` ```python import hashlib import hmac import time def verify_testingbot_signature(raw_body, timestamp, signature, secret): if int(timestamp) < int(time.time()) - 300: return False signed = f"{timestamp}.{raw_body}".encode() expected = "sha256=" + hmac.new(secret.encode(), signed, hashlib.sha256).hexdigest() return hmac.compare_digest(expected, signature or "") ``` ```csharp using System; using System.Security.Cryptography; using System.Text; public static class TestingBotWebhook { public static bool VerifySignature(string rawBody, string timestamp, string signature, string secret) { if (long.Parse(timestamp) < DateTimeOffset.UtcNow.ToUnixTimeSeconds() - 300) { return false; } using var hmac = new HMACSHA256(Encoding.UTF8.GetBytes(secret)); var digest = hmac.ComputeHash(Encoding.UTF8.GetBytes($"{timestamp}.{rawBody}")); var expected = "sha256=" + Convert.ToHexString(digest).ToLowerInvariant(); // FixedTimeEquals is a constant-time comparison. return CryptographicOperations.FixedTimeEquals( Encoding.UTF8.GetBytes(expected), Encoding.UTF8.GetBytes(signature ?? string.Empty)); } } // ASP.NET Core needs the raw body, for example: // using var reader = new StreamReader(Request.Body); // var rawBody = await reader.ReadToEndAsync(); ``` The **Test Webhook** button signs its request with the webhook's real secret when you test a saved webhook, so you can exercise your verification code from the edit page before any real test fires. After rotating the secret, update your receiver immediately: deliveries signed with the old secret no longer validate. ## Limits - You can create up to **5 webhooks per team**. - A custom payload template can be at most **64,000 characters**. - The response shown in the **Test Webhook** panel is truncated to **10,000 characters**. ## Related - [Built-in variables](https://testingbot.com/support/integrations/webhooks/built-in-variables): every variable you can use in custom payloads, URLs, headers and parameters. - [Integrations](https://testingbot.com/support/integrations/webhooks/integrations): ready-made templates for Slack, Teams, Discord, PagerDuty, OpsGenie and Jira. - [Quick Start](https://testingbot.com/support/integrations/webhooks/quick-start): create and test your first webhook step by step. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/webhooks/built-in-variables # Webhook Built-in Variables Webhook variables let you insert details of the test that triggered the webhook into the request. They work in the custom payload body, the webhook URL, header values and query parameter values. Each `{{VARIABLE}}` token is replaced with the value of the test that triggered the webhook. See [Core Concepts](https://testingbot.com/support/integrations/webhooks/core-concepts) for how payload modes, authorization, headers and parameters fit together. ## Variable reference | Variable | Description | Example | | --- | --- | --- | | `{{TEST_ID}}` | The unique session id of the test. | `f3e57e1a-sample-test-id` | | `{{TEST_NAME}}` | The name of the test, if one was set via capabilities or the API. | `Sample login test` | | `{{TEST_STATUS}}` | The result of the test: `PASSED`, `FAILED` or `UNKNOWN`. | `FAILED` | | `{{TEST_DURATION}}` | How long the test took, in seconds. Inserted as a bare number. | `42` | | `{{TEST_URL}}` | Direct link to the test details page on TestingBot. | `https://testingbot.com/members/tests/f3e57e1a-sample-test-id` | | `{{BROWSER_NAME}}` | The browser identifier the test ran on, without the version. Stable across versions, so this is the one to match on. | `googlechrome` | | `{{BROWSER_VERSION}}` | The version of the browser, as a string rather than a number, so it may carry minor or patch components. | `126` | | `{{BROWSER_LONG_VERSION}}` | The full version string when we know it, otherwise empty. | `126.0.6478.127` | | `{{BROWSER_DISPLAY_NAME}}` | A ready-made label for chat messages and alerts, with the device and OS appended on mobile. The version is already included, so pair this with nothing: use it instead of `{{BROWSER_NAME}} {{BROWSER_VERSION}}`, never alongside. | `Chrome 126` | | `{{DEVICE_NAME}}` | The device the test ran on, or empty for desktop tests. | `Galaxy S24` | | `{{PLATFORM}}` | The full name of the operating system. | `Windows 11` | | `{{BUILD_NAME}}` | The build identifier this test belongs to, or empty. | `build-1234` | | `{{TEST_TAGS}}` | Raw JSON array of tag strings attached to the test. | `["smoke","regression"]` | | `{{TEST_ERRORS}}` | Raw JSON array of error objects with `message` and `test_name` fields. | `[{"message":"Element not found","test_name":""}]` | | `{{CREATED_AT}}` | When the test started, in ISO 8601 format. | `2026-06-11T09:00:00Z` | | `{{COMPLETED_AT}}` | When the test completed, in ISO 8601 format. | `2026-06-11T09:00:42Z` | | `{{USER_ID}}` | The id of the user who launched the test. | `12345` | | `{{TEAM_ID}}` | The id of the team for which this test was run. | `12345` | ## How substitution works - In the payload body, string variables are inserted JSON-escaped without surrounding quotes; write them inside quotes yourself, for example `"name": "{{TEST_NAME}}"`. - `{{TEST_TAGS}}` and `{{TEST_ERRORS}}` insert raw JSON arrays; write them without quotes, for example `"tags": {{TEST_TAGS}}`. - `{{TEST_DURATION}}` inserts a bare number, so write it without quotes when you want a JSON number. - In the URL, header values and query parameter values, string variables are substituted as plain text; arrays are not available there. - Unknown `{{TOKENS}}` are left untouched. ## Examples The custom payload below combines quoted string variables, an unquoted number and an unquoted array: ```json { "name": "{{TEST_NAME}}", "status": "{{TEST_STATUS}}", "duration_sec": {{TEST_DURATION}}, "tags": {{TEST_TAGS}}, "link": "{{TEST_URL}}" } ``` For a failed test named `Sample login test`, the rendered request body looks like this: ```json { "name": "Sample login test", "status": "FAILED", "duration_sec": 42, "tags": ["smoke","regression"], "link": "https://testingbot.com/members/tests/f3e57e1a-sample-test-id" } ``` Note that `duration_sec` and `tags` are written without quotes in the template, so the result stays a valid JSON number and array. Use the **Test Webhook** button at [/members/integrations/webhooks](https://testingbot.com/members/integrations/webhooks) to preview the rendered payload with sample values. ## Related - [Core Concepts](https://testingbot.com/support/integrations/webhooks/core-concepts): triggers, payload modes, delivery and limits. - [Integrations](https://testingbot.com/support/integrations/webhooks/integrations): prefilled templates that use these variables for popular services. - [Quick Start](https://testingbot.com/support/integrations/webhooks/quick-start): create and test your first webhook step by step. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/webhooks/integrations # Webhook Integrations TestingBot webhooks connect your test results to the tools your team already uses. Each integration below ships with a ready-made payload template: when you create a webhook in the [members area](https://testingbot.com/members/integrations/webhooks), the **Start from a template** tiles prefill the HTTP method, URL, headers and payload body for the service you pick. A template is only a starting point: every prefilled value stays fully editable, so you can tweak the payload, add [variables](https://testingbot.com/support/integrations/webhooks/built-in-variables) or change the triggers before saving. ## Supported services [![Slack logo](https://testingbot.com/assets/integrations/slack-ba22a02abd583ab16127eed596daaa8e504ec2a1ab3c4922e3e8d19dcf052e65.svg) Slack Post a Block Kit message with the test result, browser, platform and duration to a Slack channel. ](https://testingbot.com/support/integrations/webhooks/integrations/slack) [![Microsoft Teams logo](https://testingbot.com/assets/integrations/msteams-ee9fbc29f94fe37c87ee4b76ddfbd686cb679c5f45695408454f767e8749cb3c.svg) Microsoft Teams Send a MessageCard with a facts list of test meta-data and a button that opens the test details. ](https://testingbot.com/support/integrations/webhooks/integrations/microsoft-teams) [![Discord logo](https://testingbot.com/assets/integrations/discord-ca7de98c71e3f7214b703e690c8f18cc1d16d6bdb5eae9d960575b4615125d12.svg) Discord Post a rich embed to a Discord channel, with the title linking straight to the test. ](https://testingbot.com/support/integrations/webhooks/integrations/discord) [![PagerDuty logo](https://testingbot.com/assets/integrations/pagerduty-4f2cea281aa52eb4d414e98f7b51fa5fcf7de9d8f5f32ae149d4102aa9888de8.svg) PagerDuty Trigger an incident through the Events API v2 and page your on-call engineer when a test fails. ](https://testingbot.com/support/integrations/webhooks/integrations/pagerduty) [![OpsGenie logo](https://testingbot.com/assets/integrations/opsgenie-8b1599e3a6bfd2ce362fd52a1f01c95a6e9aebc5a62f7406f8bf71ec9b5730dc.svg) OpsGenie Create an alert through the Alert API, authenticated with a prefilled GenieKey header. ](https://testingbot.com/support/integrations/webhooks/integrations/opsgenie) [![incident.io logo](https://testingbot.com/assets/integrations/incidentio-1993104f8073eee7edc3daa02d35b484a6d2ac259c5a51123049277ae53f05ed.svg) incident.io Fire an alert through the Alert Events API and let incident.io escalate or open an incident when a test fails. ](https://testingbot.com/support/integrations/webhooks/integrations/incident-io) [![Jira logo](https://testingbot.com/assets/integrations/jira-63eb4095ef566b1fe712e84f18e56b37ea706defb2b21450d93f1304c9886fa7.svg) Jira File a Bug issue through the Jira Cloud REST API v3 with the test result and a link back to the test. ](https://testingbot.com/support/integrations/webhooks/integrations/jira) Every walkthrough covers the service-side prerequisites (where to find the webhook URL or API key), the setup steps in TestingBot, the full template payload and how to test the connection. ## Build your own If your service is not listed, switch the payload toggle to **Custom** and write any JSON body you like. All [built-in variables](https://testingbot.com/support/integrations/webhooks/built-in-variables) work in the body, the URL, header values and query parameter values, so you can target any HTTP API that accepts JSON. Related: follow the [Quick Start](https://testingbot.com/support/integrations/webhooks/quick-start) to create your first webhook, or read [Core Concepts](https://testingbot.com/support/integrations/webhooks/core-concepts) for triggers, delivery and limits. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/webhooks/integrations/slack # Send Slack Messages with TestingBot Webhooks With the Slack webhook template, you get a message in your Slack channel whenever a test completes, or only when a test fails. The message shows the test result, the browser, platform and duration, plus a button that opens the test details page with video, logs and screenshots. ## Prerequisites You need a Slack [incoming webhook URL](https://api.slack.com/messaging/webhooks) for the channel where messages should appear: 1. Create a Slack app for your workspace (or open an existing one) at `api.slack.com/apps`. 2. In the app settings, open **Incoming Webhooks** and activate them. 3. Click **Add New Webhook to Workspace** and pick the target channel. 4. Copy the generated `https://hooks.slack.com/services/...` URL. Prefer a richer Slack experience? TestingBot also offers a native [Slack integration](https://testingbot.com/support/integrations/slack) with slash commands, URL unfurling and test event notifications, without writing any payloads. ## Set up the webhook in TestingBot 1. Log in to the member area and go to the [Webhooks page](https://testingbot.com/members/integrations/webhooks). 2. Click **Create webhook**. 3. Under **Start from a template** , pick the **Slack** tile. This prefills the HTTP method and the payload body. 4. Paste your `https://hooks.slack.com/services/...` URL in the URL field. 5. Adjust the triggers: **failures only** or all tests, the test type and an optional name filter. 6. Click **Create webhook** at the bottom of the form to save the configuration. ## Template payload The Slack template sends this Block Kit payload: ```json { "blocks": [ { "type": "header", "text": { "type": "plain_text", "text": "Test {{TEST_STATUS}}: {{TEST_NAME}}" } }, { "type": "section", "fields": [ { "type": "mrkdwn", "text": "*Browser:* {{BROWSER_DISPLAY_NAME}}" }, { "type": "mrkdwn", "text": "*Platform:* {{PLATFORM}}" }, { "type": "mrkdwn", "text": "*Duration:* {{TEST_DURATION}} seconds" } ] }, { "type": "actions", "elements": [ { "type": "button", "text": { "type": "plain_text", "text": "View test" }, "url": "{{TEST_URL}}" } ] } ] } ``` There are no placeholders to replace in this payload: the Slack channel is determined by the webhook URL itself. The `{{...}}` tokens are substituted with the values of the test that triggered the webhook; the full list is on the [built-in variables](https://testingbot.com/support/integrations/webhooks/built-in-variables) page. ## Test it Click **Test Webhook** in the form to send a sample request with example values. On success, a message appears in your Slack channel within a few seconds and the Response panel shows a `200` status with the body `ok`. If nothing arrives, check for these common causes: - The webhook URL is incomplete or was revoked in Slack. - Incoming webhooks are not activated for your Slack app. - The payload JSON was edited and is no longer valid; Slack responds with `invalid_payload`. Related: [Webhooks Quick Start](https://testingbot.com/support/integrations/webhooks/quick-start), [all webhook templates](https://testingbot.com/support/integrations/webhooks/integrations) and the [Microsoft Teams walkthrough](https://testingbot.com/support/integrations/webhooks/integrations/microsoft-teams). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/webhooks/integrations/microsoft-teams # Send Microsoft Teams Notifications with TestingBot Webhooks With the Microsoft Teams webhook template, a card appears in your Teams channel whenever a test completes, or only when one fails. The card lists the browser, platform, duration and status of the test, with a **View test** button that opens the full test report on TestingBot. ## Prerequisites You need an incoming webhook URL for the Teams channel that should receive the notifications: 1. In Microsoft Teams, open the channel, click the **...** menu and choose **Connectors** (or **Manage channel** \> **Connectors** ). 2. Find **Incoming Webhook** , click **Configure** , give it a name and optionally an icon. 3. Click **Create** and copy the generated webhook URL. If your organization has retired Office 365 connectors, you can create the incoming webhook with the **Workflows** app in Teams instead; the resulting URL works the same way. ## Set up the webhook in TestingBot 1. Log in to the member area and go to the [Webhooks page](https://testingbot.com/members/integrations/webhooks). 2. Click **Create webhook**. 3. Under **Start from a template** , pick the **Microsoft Teams** tile. This prefills the HTTP method and the MessageCard payload. 4. Paste your Teams webhook URL in the URL field. 5. Adjust the triggers: **failures only** or all tests, the test type and an optional name filter. 6. Click **Create webhook** at the bottom of the form to save the configuration. ## Template payload The Microsoft Teams template sends this MessageCard payload: ```json { "@type": "MessageCard", "@context": "https://schema.org/extensions", "themeColor": "0076D7", "summary": "Test {{TEST_STATUS}}: {{TEST_NAME}}", "title": "Test {{TEST_STATUS}}: {{TEST_NAME}}", "sections": [ { "facts": [ { "name": "Browser", "value": "{{BROWSER_DISPLAY_NAME}}" }, { "name": "Platform", "value": "{{PLATFORM}}" }, { "name": "Duration", "value": "{{TEST_DURATION}} seconds" }, { "name": "Status", "value": "{{TEST_STATUS}}" } ] } ], "potentialAction": [ { "@type": "OpenUri", "name": "View test", "targets": [ { "os": "default", "uri": "{{TEST_URL}}" } ] } ] } ``` There are no placeholders to replace: the target channel is determined by the webhook URL. You can change the `themeColor` or add extra facts; the `{{...}}` tokens are documented on the [built-in variables](https://testingbot.com/support/integrations/webhooks/built-in-variables) page. ## Test it Click **Test Webhook** in the form to send a sample card right away. On success, the card shows up in your Teams channel and the Response panel reports a `200` status, usually with the body `1`. Common failure causes: - The webhook URL was removed or the connector was deleted from the channel. - Your organization blocks incoming webhooks; ask a Teams administrator, or use the Workflows app. - The payload was edited into invalid JSON or an invalid MessageCard; Teams then returns a 4xx status with an error summary. Related: [Webhooks Quick Start](https://testingbot.com/support/integrations/webhooks/quick-start), [all webhook templates](https://testingbot.com/support/integrations/webhooks/integrations) and the [Slack walkthrough](https://testingbot.com/support/integrations/webhooks/integrations/slack). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/webhooks/integrations/discord # Send Discord Messages with TestingBot Webhooks With the Discord webhook template, an embed is posted to your Discord channel whenever a test completes, or only when a test fails. The embed title links straight to the test details page, with inline fields for the browser, platform and duration. ## Prerequisites You need a webhook URL for the Discord channel that should receive the messages: 1. In Discord, open the channel settings (the gear icon next to the channel name). 2. Go to **Integrations** \> **Webhooks**. 3. Click **New Webhook** , give it a name and pick the channel. 4. Click **Copy Webhook URL**. You need the **Manage Webhooks** permission on the server to create channel webhooks. ## Set up the webhook in TestingBot 1. Log in to the member area and go to the [Webhooks page](https://testingbot.com/members/integrations/webhooks). 2. Click **Create webhook**. 3. Under **Start from a template** , pick the **Discord** tile. This prefills the HTTP method and the embed payload. 4. Paste your `https://discord.com/api/webhooks/...` URL in the URL field. 5. Adjust the triggers: **failures only** or all tests, the test type and an optional name filter. 6. Click **Create webhook** at the bottom of the form to save the configuration. ## Template payload The Discord template sends this embed payload: ```json { "embeds": [ { "title": "Test {{TEST_STATUS}}: {{TEST_NAME}}", "url": "{{TEST_URL}}", "description": "The test finished with status {{TEST_STATUS}}.", "fields": [ { "name": "Browser", "value": "{{BROWSER_DISPLAY_NAME}}", "inline": true }, { "name": "Platform", "value": "{{PLATFORM}}", "inline": true }, { "name": "Duration", "value": "{{TEST_DURATION}} seconds", "inline": true } ] } ] } ``` There are no placeholders to replace: the channel is determined by the webhook URL. You can customize the embed, for example add a `color` field; the `{{...}}` tokens are listed on the [built-in variables](https://testingbot.com/support/integrations/webhooks/built-in-variables) page. ## Test it Click **Test Webhook** in the form to send a sample embed right away. On success, the embed appears in your Discord channel and the Response panel shows a `204 No Content` status with an empty body, which is how Discord acknowledges a delivery. Common failure causes: - The webhook was deleted in the channel settings, which returns a `404 Unknown Webhook` error. - The URL is incomplete, for example the token part after the webhook id is missing. - The payload was edited into invalid JSON or an invalid embed; Discord returns a `400` with validation details in the response body. Related: [Webhooks Quick Start](https://testingbot.com/support/integrations/webhooks/quick-start), [all webhook templates](https://testingbot.com/support/integrations/webhooks/integrations) and the [Core Concepts](https://testingbot.com/support/integrations/webhooks/core-concepts) page. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/webhooks/integrations/pagerduty # Create PagerDuty Incidents from Failed Tests With the PagerDuty webhook template, TestingBot opens an incident in PagerDuty the moment a test fails. Your on-call engineer is paged with the test name, browser, platform and a direct link to the test details page, so the failure can be investigated right away. The template uses the [PagerDuty Events API v2](https://developer.pagerduty.com/docs/events-api-v2-overview): you only need a routing key, no PagerDuty user account credentials are involved. ## Prerequisites You need an Events API v2 routing key from PagerDuty: 1. In PagerDuty, open the **service** that should receive the incidents (or create a new one). 2. On the service page, go to the **Integrations** tab and add an **Events API v2** integration. 3. Copy the **Integration Key** (also called the routing key) that PagerDuty generates. The webhook URL is always the fixed PagerDuty endpoint `https://events.pagerduty.com/v2/enqueue`; the routing key inside the payload determines which service receives the event. ## Set up the webhook in TestingBot 1. Log in to the TestingBot member area and open the [Webhooks page](https://testingbot.com/members/integrations/webhooks) under Integrations. 2. Click **Create webhook** and give it a name, for example `PagerDuty incidents`. 3. Under **Start from a template** , pick the **PagerDuty** tile. This prefills the URL `https://events.pagerduty.com/v2/enqueue`, the POST method and the payload below. 4. In the **Body** tab, replace `YOUR_PAGERDUTY_ROUTING_KEY` with the Integration Key you copied from PagerDuty. 5. Set the trigger to **failures only** , so an incident is created only when a test fails. Optionally narrow it down with a test type or a name filter such as `regression_*`. 6. Click **Create webhook** at the bottom of the form to save the configuration. ## Template payload The PagerDuty template sends this JSON body. Replace `YOUR_PAGERDUTY_ROUTING_KEY` with your own routing key; the `{{...}}` tokens are substituted with test values at delivery time and are documented on the [built-in variables](https://testingbot.com/support/integrations/webhooks/built-in-variables) page. ```json { "routing_key": "YOUR_PAGERDUTY_ROUTING_KEY", "event_action": "trigger", "payload": { "summary": "Test {{TEST_STATUS}}: {{TEST_NAME}}", "source": "TestingBot", "severity": "error", "custom_details": { "browser": "{{BROWSER_DISPLAY_NAME}}", "platform": "{{PLATFORM}}", "duration_sec": {{TEST_DURATION}}, "tags": {{TEST_TAGS}} } }, "links": [ { "href": "{{TEST_URL}}", "text": "View test on TestingBot" } ] } ``` The incident summary contains the test result and name, the custom details carry the browser, platform, duration and tags, and the attached link opens the test on TestingBot. Everything stays editable: you can change the `severity`, add a `dedup_key` or extend `custom_details` with more variables. ## Test it Click **Test Webhook** in the form to send a sample event with placeholder test values. The response panel shows the HTTP status, duration, headers and body returned by PagerDuty; a successful request returns a 2xx status with `"status": "success"` in the body, and a new incident appears on the PagerDuty service you connected. If no incident shows up, check these common causes: - The routing key is wrong or still set to the `YOUR_PAGERDUTY_ROUTING_KEY` placeholder; PagerDuty responds with a 4xx status in that case. - The integration was created on a different service than the one you are watching. - The service in PagerDuty is disabled or in maintenance mode. ## Related - [Built-in variables](https://testingbot.com/support/integrations/webhooks/built-in-variables): every token you can use in the payload, URL, headers and parameters. - [OpsGenie webhooks](https://testingbot.com/support/integrations/webhooks/integrations/opsgenie): open alerts in OpsGenie instead of PagerDuty. - [incident.io webhooks](https://testingbot.com/support/integrations/webhooks/integrations/incident-io): fire alerts in incident.io instead of PagerDuty. - [Core concepts](https://testingbot.com/support/integrations/webhooks/core-concepts): triggers, payload modes, delivery and limits. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/webhooks/integrations/opsgenie # Open OpsGenie Alerts when Tests Fail With the OpsGenie webhook template, TestingBot creates an alert in OpsGenie whenever a test completes, or only when it fails. The alert carries the test name, status, browser, platform and a link to the test details page, so the on-call responder sees everything needed to triage the failure. The template posts to the [OpsGenie Alert API](https://docs.opsgenie.com/docs/alert-api) and authenticates with an API key in the Authorization header. ## Prerequisites You need an OpsGenie API key from an API integration: 1. In OpsGenie, go to **Settings** and open **Integrations** (or open the team that should receive the alerts and use its Integrations tab). 2. Add an **API** integration and make sure it has **create alert** access. 3. Copy the **API key** that OpsGenie generates for the integration. The key is sent in an Authorization header with the value `GenieKey YOUR_OPSGENIE_API_KEY`. The webhook URL is the fixed endpoint `https://api.opsgenie.com/v2/alerts`. If your OpsGenie account is hosted in the EU region, use `https://api.eu.opsgenie.com/v2/alerts` instead. ## Set up the webhook in TestingBot 1. Log in to the TestingBot member area and open the [Webhooks page](https://testingbot.com/members/integrations/webhooks) under Integrations. 2. Click **Create webhook** and give it a name, for example `OpsGenie alerts`. 3. Under **Start from a template** , pick the **OpsGenie** tile. This prefills the URL `https://api.opsgenie.com/v2/alerts`, the POST method, the payload below and an `Authorization` header in the **Headers** tab. 4. In the **Headers** tab, replace `YOUR_OPSGENIE_API_KEY` in the header value `GenieKey YOUR_OPSGENIE_API_KEY` with the API key you copied. 5. Set the trigger to **failures only** if you only want alerts for failing tests. Optionally narrow it down with a test type or a name filter such as `regression_*`. 6. Click **Create webhook** at the bottom of the form to save the configuration. ## Template payload The OpsGenie template sends this JSON body. There are no placeholders to edit inside the body itself: your API key lives in the Authorization header instead. The `{{...}}` tokens are substituted with test values at delivery time and are documented on the [built-in variables](https://testingbot.com/support/integrations/webhooks/built-in-variables) page. ```json { "message": "Test {{TEST_STATUS}}: {{TEST_NAME}}", "description": "Test {{TEST_NAME}} finished with status {{TEST_STATUS}} on {{PLATFORM}}. View it at {{TEST_URL}}", "priority": "P3", "details": { "test_id": "{{TEST_ID}}", "browser": "{{BROWSER_DISPLAY_NAME}}", "platform": "{{PLATFORM}}", "test_url": "{{TEST_URL}}" } } ``` The alert message shows the test result and name, the description includes the platform and a link, and the details section adds structured meta-data. Everything stays editable: you can raise the `priority` to `P1`, add `tags` or `responders`, or extend `details` with more variables. ## Test it Click **Test Webhook** in the form to send a sample alert with placeholder test values. The response panel shows the HTTP status, duration, headers and body returned by OpsGenie; a successful request returns a 202 status with a `requestId` in the body, and a new alert appears in your OpsGenie alert list. If no alert shows up, check these common causes: - The Authorization header still contains the `YOUR_OPSGENIE_API_KEY` placeholder, or the key is wrong; OpsGenie responds with a 401 or 403 status in that case. - The API integration is disabled or lacks create alert access. - Your account is in the EU region but the webhook still points at the US endpoint. ## Related - [Built-in variables](https://testingbot.com/support/integrations/webhooks/built-in-variables): every token you can use in the payload, URL, headers and parameters. - [PagerDuty webhooks](https://testingbot.com/support/integrations/webhooks/integrations/pagerduty): open incidents in PagerDuty instead of OpsGenie. - [incident.io webhooks](https://testingbot.com/support/integrations/webhooks/integrations/incident-io): fire alerts in incident.io instead of OpsGenie. - [Core concepts](https://testingbot.com/support/integrations/webhooks/core-concepts): triggers, payload modes, delivery and limits. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/webhooks/integrations/incident-io # Create incident.io Alerts from Failed Tests With the incident.io webhook template, TestingBot fires an alert in [incident.io](https://incident.io) the moment a test fails. From there, incident.io takes over: the alert is grouped and routed by your alert configuration, can page the on-call engineer through an escalation path, and can create an incident with the test name, browser, platform and a direct link to the test details page. The template uses the [incident.io Alert Events API](https://api-docs.incident.io/tag/Alert-Events-V2) through an HTTP alert source: you only need the alert source URL and its secret, no incident.io user account credentials are involved. ## Prerequisites You need an HTTP alert source in incident.io: 1. In incident.io, go to **Alerts** → **Configure** and create a new alert source of type **HTTP**. Pick the **Default** payload option, name it for example `TestingBot`. 2. incident.io shows the endpoint URL for the source, in the form `https://api.incident.io/v2/alert_events/http/`. Copy it. 3. Copy the **secret** (token) that incident.io generated for the source. It is used as a Bearer token when TestingBot delivers the alert. 4. Connect the alert source to an **alert route** so incoming alerts create escalations or incidents. Without a route, alerts arrive but nothing is paged. ## Set up the webhook in TestingBot 1. Log in to the TestingBot member area and open the [Webhooks page](https://testingbot.com/members/integrations/webhooks) under Integrations. 2. Click **Create webhook** and give it a name, for example `incident.io alerts`. 3. Under **Start from a template** , pick the **incident.io** tile. This prefills the POST method, the Authorization header and the payload below. 4. Replace `YOUR_ALERT_SOURCE_CONFIG_ID` in the URL with the id from the endpoint URL you copied from incident.io (or paste the full URL). 5. In the **Headers** tab, replace `YOUR_ALERT_SOURCE_SECRET` with the secret of your alert source, keeping the `Bearer ` prefix. 6. Set the trigger to **failures only** , so an alert fires only when a test fails. Optionally narrow it down with a test type or a name filter such as `regression_*`. 7. Click **Create webhook** at the bottom of the form to save the configuration. ## Template payload The incident.io template sends this JSON body. The `{{...}}` tokens are substituted with test values at delivery time and are documented on the [built-in variables](https://testingbot.com/support/integrations/webhooks/built-in-variables) page. ```json { "title": "Test {{TEST_STATUS}}: {{TEST_NAME}}", "description": "Test {{TEST_NAME}} finished with status {{TEST_STATUS}} on {{PLATFORM}}. View it at {{TEST_URL}}", "status": "firing", "deduplication_key": "{{TEST_ID}}", "source_url": "{{TEST_URL}}", "metadata": { "browser": "{{BROWSER_DISPLAY_NAME}}", "platform": "{{PLATFORM}}", "build": "{{BUILD_NAME}}", "duration_sec": {{TEST_DURATION}}, "tags": {{TEST_TAGS}} } } ``` The alert title contains the test result and name, the `source_url` opens the test on TestingBot, and the `metadata` object carries the browser, platform, build, duration and tags. In incident.io you can map metadata fields to alert attributes and priorities, for example routing by tag or team. Everything stays editable. One useful tweak: change the `deduplication_key` to `{{TEST_NAME}}` so repeated failures of the same test collapse into a single alert instead of paging again for every run. ## Auto-resolving alerts The template always sends `"status": "firing"`, so alerts are opened but never closed by TestingBot. If you want an alert to resolve automatically when the test passes again, use a **Custom** HTTP alert source in incident.io instead of a Default one: 1. In the TestingBot webhook, change the payload to `"status": "{{TEST_STATUS}}"`, set the `deduplication_key` to `{{TEST_NAME}}` and set the trigger to **all tests** instead of failures only. 2. In incident.io, create the HTTP alert source with the **Custom** payload option and use a transform expression that maps the TestingBot status onto the alert status: ```javascript return { title: $.title, description: $.description, status: $.status === "PASSED" ? "resolved" : "firing", source_url: $.source_url, metadata: $.metadata, }; ``` Set the deduplication key path of the source to `deduplication_key`. A failing test now opens an alert, and the next passing run of the same test resolves it. ## Test it Click **Test Webhook** in the form to send a sample event with placeholder test values. The response panel shows the HTTP status, duration, headers and body returned by incident.io; a successful request returns a `202 Accepted` status with `"status": "success"` in the body, and a new alert appears on the alert source in incident.io. If no alert shows up, check these common causes: - The Authorization header is wrong or still set to the `YOUR_ALERT_SOURCE_SECRET` placeholder; incident.io responds with a 401 status in that case. - The URL still contains the `YOUR_ALERT_SOURCE_CONFIG_ID` placeholder, or the id belongs to a different alert source. - The alert arrived but is not connected to an alert route, so no escalation or incident was created. Check the alert source's recent events in incident.io. ## Related - [Built-in variables](https://testingbot.com/support/integrations/webhooks/built-in-variables): every token you can use in the payload, URL, headers and parameters. - [PagerDuty webhooks](https://testingbot.com/support/integrations/webhooks/integrations/pagerduty): page your on-call engineer through PagerDuty instead. - [OpsGenie webhooks](https://testingbot.com/support/integrations/webhooks/integrations/opsgenie): open alerts in OpsGenie instead. - [Core concepts](https://testingbot.com/support/integrations/webhooks/core-concepts): triggers, payload modes, delivery and limits. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/webhooks/integrations/jira # File Jira Bugs Automatically for Failed Tests With the Jira webhook template, TestingBot creates a Bug issue in your Jira Cloud project whenever a test fails. The issue summary contains the test result and name, and the description links straight to the test details page with its video, logs and screenshots. The template posts to the [Jira Cloud REST API v3](https://developer.atlassian.com/cloud/jira/platform/rest/v3/api-group-issues/) and authenticates with Basic Auth. TestingBot also offers a native [Jira integration](https://testingbot.com/support/integrations/jira) for manually reporting bugs from test results; the webhook described here is the fully automated alternative. ## Prerequisites You need an Atlassian API token and a few details about your Jira site: 1. Create an [Atlassian API token](https://id.atlassian.com/manage-profile/security/api-tokens) and copy it. The token is paired with your Atlassian account email for Basic Auth. 2. Note your Jira site domain: the webhook URL is `https://YOUR-DOMAIN.atlassian.net/rest/api/3/issue` with your own domain filled in. 3. Note the **project key** of the project where issues should be created, for example `QA`, and make sure the project has a **Bug** issue type (or another issue type you want to use). The account that owns the API token needs permission to create issues in that project. ## Set up the webhook in TestingBot 1. Log in to the TestingBot member area and open the [Webhooks page](https://testingbot.com/members/integrations/webhooks) under Integrations. 2. Click **Create webhook** and give it a name, for example `Jira bugs`. 3. Under **Start from a template** , pick the **Jira** tile. This prefills the URL, the POST method and the payload below. 4. In the URL, replace `YOUR-DOMAIN` with your Jira site domain. 5. In the **Authorization** tab, choose **Basic Auth** , enter your Atlassian account email as the username and the API token as the password. 6. In the **Body** tab, replace `YOUR_PROJECT_KEY` with your project key, and change the `issuetype` name if your project does not use `Bug`. 7. Set the trigger to **failures only** , so an issue is only filed when a test fails. Optionally narrow it down with a test type or a name filter such as `regression_*`. 8. Click **Create webhook** at the bottom of the form to save the configuration. ## Template payload The Jira template sends this JSON body. Replace `YOUR_PROJECT_KEY` with the key of your Jira project; the `{{...}}` tokens are substituted with test values at delivery time and are documented on the [built-in variables](https://testingbot.com/support/integrations/webhooks/built-in-variables) page. ```json { "fields": { "project": { "key": "YOUR_PROJECT_KEY" }, "issuetype": { "name": "Bug" }, "summary": "Test {{TEST_STATUS}}: {{TEST_NAME}}", "description": { "type": "doc", "version": 1, "content": [ { "type": "paragraph", "content": [ { "type": "text", "text": "Test {{TEST_NAME}} finished with status {{TEST_STATUS}} on {{PLATFORM}}. View it at {{TEST_URL}}" } ] } ] } } } ``` The description uses the Atlassian Document Format (ADF), which the Jira Cloud REST API v3 requires. Everything stays editable: you can add fields such as `labels` or `priority`, or extend the description with more paragraphs and variables, as long as the ADF structure stays valid. ## Test it Click **Test Webhook** in the form to send a sample request with placeholder test values. The response panel shows the HTTP status, duration, headers and body returned by Jira; a successful request returns a 201 status with the new issue key in the body, and the issue appears in your project. If no issue is created, check these common causes: - The Basic Auth credentials are wrong: use your Atlassian email as username and the API token (not your password) as password. Jira responds with a 401 status in that case. - The project key is wrong or still set to the `YOUR_PROJECT_KEY` placeholder, or the account lacks create permission; Jira responds with a 400 or 403 status and an error message in the body. - The issue type `Bug` does not exist in the project, or a required custom field is missing from the payload. ## Related - [Native Jira integration](https://testingbot.com/support/integrations/jira): report bugs manually from a test result page. - [Built-in variables](https://testingbot.com/support/integrations/webhooks/built-in-variables): every token you can use in the payload, URL, headers and parameters. - [Core concepts](https://testingbot.com/support/integrations/webhooks/core-concepts): triggers, payload modes, delivery and limits. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/ci-cd # Continuous Integration Continuous Integration makes it easy to run tests at a given interval, after committing new code, before deploying changes, etc. There are various providers offering Continuous Integration, both cloud-based services and do-it-yourself services. If you're looking to host your own CI, here are some options: [Jenkins](https://jenkins.io/), [Bamboo](https://www.atlassian.com/software/bamboo) and [TeamCity](https://www.jetbrains.com/teamcity/). ## Example CI Integrations with TestingBot Please follow a tutorial below to connect your tests with TestingBot and your CI of choice: ### Cloud CI providers ![AWS CodePipeline](https://testingbot.com/assets/integrations/aws-37fe010e66887d20cd831130e90da33a0ec731b5dd87fbba8adc933db17b0709.webp) ### [AWS CodePipeline](https://testingbot.com/support/integrations/ci-cd/aws-codepipeline) CodeBuild buildspec with tunnel ![Azure DevOps (VSTS)](https://testingbot.com/assets/integrations/devops-7de50ed101385e09ac3fb0b1110a601e229282512a1b25460fdaedd22ef244bb.webp) ### [Azure DevOps (VSTS)](https://testingbot.com/support/integrations/ci-cd/azure) Integrate with VSTS ![Bitbucket Pipelines](https://testingbot.com/assets/integrations/bitbucket-02aabcd549221726e052f2ee19c8959fbb218160ad7f7c31f6e6b96e7824a42c.webp) ### [Bitbucket Pipelines](https://testingbot.com/support/integrations/ci-cd/bitbucket) Bitbucket Pipelines integration ![Bitrise](https://testingbot.com/assets/integrations/bitrise-d122449f8ba4609306287a7080cef27601f50d50b7bdf23811751de4d5ea41d6.svg) ### [Bitrise](https://testingbot.com/support/integrations/ci-cd/bitrise) Mobile app CI/CD integration ![Buildkite](https://testingbot.com/assets/integrations/buildkite-d573ce24dc0bb8ce8df123e751563dd8ee85b93dcdaba4a49d72ff0a678d34f6.svg) ### [Buildkite](https://testingbot.com/support/integrations/ci-cd/buildkite) Tunnel, statuses and annotations ![CircleCI](https://testingbot.com/assets/integrations/circleci-ede029654ff39b60342e4a5bba8fc9a3a49305cc38bda215b7cf8d07e85e0543.webp) ### [CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci) Official orb with tunnel support ![GitHub Actions](https://testingbot.com/assets/integrations/github-actions-8436242ad9e15381f84f1a98a10dddce15611087f121d6305782e211942b7842.webp) ### [GitHub Actions](https://testingbot.com/support/integrations/ci-cd/github-actions) Launch tests from workflows ![GitHub PR Checks](https://testingbot.com/assets/integrations/github-d03b3ecd2162b559b5160a1e89db53b782a287ae0402f80469c7d5be1e54646b.svg) ### [GitHub PR Checks](https://testingbot.com/support/integrations/ci-cd/github-pr-checks) Status checks on pull requests ![GitLab CI](https://testingbot.com/assets/integrations/gitlab-e4c0b6db07d07bc0a400f5606e258f4dc8a51b778e97bbfa51d1d531f3cb049a.webp) ### [GitLab CI](https://testingbot.com/support/integrations/ci-cd/gitlab) Integrate with GitLab CI ![Travis CI](https://testingbot.com/assets/integrations/travis-f3a8872deab0dd6f4c13686fd8255cb07c426309090e335a9198312b49c43232.webp) ### [Travis CI](https://testingbot.com/support/integrations/ci-cd/travis-ci) Integrate with Travis CI ### Self-hosted CI servers ![Jenkins CI](https://testingbot.com/assets/integrations/jenkins-e586a1c4af833e4c8d38d7b392a4e84bd193f30cae9cf8c9f198a58ec943a1ff.webp) ### [Jenkins CI](https://testingbot.com/support/integrations/ci-cd/jenkins) Pipeline and tunnel support ![Bamboo](https://testingbot.com/assets/integrations/bamboo-c8fe09a3cb0b74d6055e3aa6607d77720bfe29ff487b8e3cf825138adf634346.svg) ### [Bamboo](https://testingbot.com/support/integrations/ci-cd/bamboo) Atlassian Bamboo plugin ![TeamCity CI](https://testingbot.com/assets/integrations/teamcity-7e75cdb835903a4983ad276c41d95e569ceac5be486e96ec1f955029ce5bb319.svg) ### [TeamCity CI](https://testingbot.com/support/integrations/ci-cd/teamcity) Plugin for JetBrains TeamCity ### Build tools ![Gradle](https://testingbot.com/assets/integrations/gradle-a7d0123cadbbf7d2bf598be31e0b4cf9f035c9924f988603d5b8a533385ef352.svg) ### [Gradle](https://testingbot.com/support/integrations/ci-cd/gradle) Espresso tests from your build ## CI Plugins We also provide a Jenkins plugin that makes integrating your tests with TestingBot on Jenkins very easy. It allows you to view your test results straight from inside Jenkins. See [how to install this Jenkins plugin](https://testingbot.com/support/integrations/ci-cd/jenkins). For Android, our [Gradle plugin](https://testingbot.com/support/integrations/ci-cd/gradle) uploads your app and runs Espresso tests on our real device cloud directly from your build. For Buildkite, our [Buildkite plugin](https://testingbot.com/support/integrations/ci-cd/buildkite) starts a TestingBot Tunnel around your build step, updates test statuses and annotates the build with your TestingBot test results. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/ci-cd/aws-codepipeline # AWS CodePipeline Automated Testing [AWS CodePipeline](https://aws.amazon.com/codepipeline/) is a continuous delivery service that models your release process as a set of stages: a source stage (GitHub, Bitbucket, CodeCommit or Amazon S3), a build stage and one or more deploy stages. Your tests run in the build stage, which is usually handled by [AWS CodeBuild](https://aws.amazon.com/codebuild/). CodeBuild reads a `buildspec.yml` file from the root of your repository: this is where you install dependencies, start a [TestingBot Tunnel](https://testingbot.com/support/tunnel) if needed, and run your Selenium or Appium tests on the TestingBot browser and device cloud. If you already have tests running locally, only two things are needed: make your TestingBot key and secret available to the build, and point your tests at `https://hub.testingbot.com/wd/hub`. ## Prerequisites - An AWS account with permission to create CodePipeline and CodeBuild projects. - A repository with your tests, connected to CodePipeline as a source. - A `buildspec.yml` file in the root of that repository. - A TestingBot key and secret, available in the [TestingBot member area](https://testingbot.com/members/user/edit). ## Credentials Your tests read the TestingBot key and secret from the `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` environment variables. There are three ways to provide them to CodeBuild, in order of preference: ### 1. AWS Secrets Manager (recommended) Store your credentials as a secret, for example a secret named `testingbot` with the keys `key` and `secret`, then reference it from your `buildspec.yml`: ```yaml env: secrets-manager: TESTINGBOT_KEY: testingbot:key TESTINGBOT_SECRET: testingbot:secret ``` The CodeBuild service role needs `secretsmanager:GetSecretValue` permission on that secret. The values never appear in your repository and CodeBuild masks them in the build log. ### 2. AWS Systems Manager Parameter Store ```yaml env: parameter-store: TESTINGBOT_KEY: /testingbot/key TESTINGBOT_SECRET: /testingbot/secret ``` Store both as `SecureString` parameters and grant the build role `ssm:GetParameters`. ### 3. Plain environment variables You can also set the variables directly on the CodeBuild project, under **Environment** → **Additional configuration** → **Environment variables** , or in the build stage of the Create Pipeline wizard. ```yaml env: variables: TESTINGBOT_KEY: "YOUR_KEY" TESTINGBOT_SECRET: "YOUR_SECRET" ``` Do not commit your TestingBot secret to your repository. Plain `env.variables` entries in `buildspec.yml` are visible to everyone with access to the repository and to the build logs. Use Secrets Manager or Parameter Store instead. ## Create a Pipeline In the AWS console, go to **CodePipeline** and choose **Create pipeline** : 1. **Pipeline settings** : give the pipeline a name and let AWS create a new service role. 2. **Source stage** : pick your source provider (GitHub, Bitbucket, CodeCommit or S3), the repository and the branch that should trigger the pipeline. 3. **Build stage** : choose **AWS CodeBuild** as the build provider and select your build project. It helps to create the CodeBuild project beforehand, because its name is what you will recognize in your build logs. Add the TestingBot environment variables here if you are not using Secrets Manager. 4. **Deploy stage** : optional, select a deploy provider if your pipeline also deploys. 5. Review the configuration and choose **Create pipeline**. From now on, every commit to the configured branch triggers the pipeline and runs your tests on TestingBot. ## Buildspec Example Below is a `buildspec.yml` that installs the project dependencies and runs a test suite against the TestingBot cloud: ```yaml version: 0.2 env: secrets-manager: TESTINGBOT_KEY: testingbot:key TESTINGBOT_SECRET: testingbot:secret phases: install: runtime-versions: nodejs: 20 commands: - npm ci build: commands: - npm run test:e2e reports: e2e: files: - "junit/*.xml" ``` The optional `reports` section makes CodeBuild display your JUnit results in the AWS console, next to the full test reports, videos and logs on TestingBot. ## Locally Hosted Websites If the website you want to test is not publicly reachable (a staging environment inside a VPC, or a server started by the build itself), start a [TestingBot Tunnel](https://testingbot.com/support/tunnel) in the `pre_build` phase. The tunnel is a Java application, so add a Java runtime to the build image. ```yaml version: 0.2 env: secrets-manager: TESTINGBOT_KEY: testingbot:key TESTINGBOT_SECRET: testingbot:secret phases: install: runtime-versions: nodejs: 20 java: corretto17 commands: - npm ci pre_build: commands: - curl -sS -O https://testingbot.com/downloads/testingbot-tunnel.zip - unzip -o -q testingbot-tunnel.zip - | java -jar testingbot-tunnel.jar $TESTINGBOT_KEY $TESTINGBOT_SECRET \ --tunnel-identifier codebuild-$CODEBUILD_BUILD_NUMBER \ --readyfile /tmp/tunnel.ready \ --logfile /tmp/tunnel.log & - timeout 180 bash -c 'until [-f /tmp/tunnel.ready]; do sleep 1; done' build: commands: - npm run start:staging & - npm run test:e2e post_build: commands: - pkill -f testingbot-tunnel || true artifacts: files: - /tmp/tunnel.log ``` A few things worth pointing out: - The `--readyfile` flag touches a file as soon as the tunnel is ready. Waiting for that file is more reliable than a fixed `sleep`, because your tests never start before the tunnel is up. - The `--tunnel-identifier` flag tags the tunnel, so parallel builds each get their own tunnel. Pass the same value as the `tunnel-identifier` capability in your tests. `CODEBUILD_BUILD_NUMBER` is set by CodeBuild for every build. - The `post_build` phase always runs, also when your tests fail, which makes it the right place to stop the tunnel. - Uploading `/tmp/tunnel.log` as an artifact makes it easy to debug connection problems afterwards. If your staging environment lives in a private VPC, configure the CodeBuild project to run inside that VPC. The tunnel then reaches your internal hosts the same way the build does. Instead of the tunnel identifier, you can also point your tests at the local relay the tunnel exposes on `http://localhost:4445/wd/hub`. See the [tunnel command line options](https://testingbot.com/support/tunnel/commandline) for all available flags. ## Example Test Your test reads the credentials from the environment and sends the TestingBot capabilities in the `tb:options` object: [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) ```ruby #!/usr/bin/env ruby require 'rubygems' require 'selenium-webdriver' options = Selenium::WebDriver::Chrome::Options.new options.add_option('platformName', 'WIN11') options.add_option('browserVersion', 'latest') options.add_option('tb:options', { 'key' => ENV['TESTINGBOT_KEY'], 'secret' => ENV['TESTINGBOT_SECRET'], 'name' => 'CodePipeline Test', 'build' => "codebuild-#{ENV['CODEBUILD_BUILD_NUMBER']}", 'tunnel-identifier' => "codebuild-#{ENV['CODEBUILD_BUILD_NUMBER']}" }) driver = Selenium::WebDriver.for( :remote, url: "https://hub.testingbot.com/wd/hub", options: options) driver.navigate.to "http://localhost:8080" puts driver.title driver.quit ``` ```python import os from selenium import webdriver from selenium.webdriver.chrome.options import Options build = "codebuild-%s" % os.environ.get('CODEBUILD_BUILD_NUMBER') options = Options() options.set_capability('platformName', 'WIN11') options.set_capability('browserVersion', 'latest') options.set_capability('tb:options', { 'key': os.environ['TESTINGBOT_KEY'], 'secret': os.environ['TESTINGBOT_SECRET'], 'name': 'CodePipeline Test', 'build': build, 'tunnel-identifier': build }) driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options ) driver.get("http://localhost:8080") print(driver.title) driver.quit() ``` ```java import org.openqa.selenium.WebDriver; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URL; import java.util.HashMap; import java.util.Map; public class CodePipelineTest { public static final String KEY = System.getenv("TESTINGBOT_KEY"); public static final String SECRET = System.getenv("TESTINGBOT_SECRET"); public static final String URL = "https://hub.testingbot.com/wd/hub"; public static void main(String[] args) throws Exception { String build = "codebuild-" + System.getenv("CODEBUILD_BUILD_NUMBER"); ChromeOptions options = new ChromeOptions(); options.setCapability("browserVersion", "latest"); options.setCapability("platformName", "WIN11"); Map tbOptions = new HashMap<>(); tbOptions.put("key", KEY); tbOptions.put("secret", SECRET); tbOptions.put("name", "CodePipeline Test"); tbOptions.put("build", build); tbOptions.put("tunnel-identifier", build); options.setCapability("tb:options", tbOptions); WebDriver driver = new RemoteWebDriver(new URL(URL), options); driver.get("http://localhost:8080"); System.out.println(driver.getTitle()); driver.quit(); } } ``` ```javascript const webdriver = require('selenium-webdriver'); const chrome = require('selenium-webdriver/chrome'); const build = `codebuild-${process.env.CODEBUILD_BUILD_NUMBER}`; async function runCodePipelineTest () { let options = new chrome.Options(); options.set('platformName', 'WIN11'); options.set('browserVersion', 'latest'); options.set('tb:options', { 'key': process.env.TESTINGBOT_KEY, 'secret': process.env.TESTINGBOT_SECRET, 'name': 'CodePipeline Test', 'build': build, 'tunnel-identifier': build }); let driver = new webdriver.Builder() .usingServer('https://hub.testingbot.com/wd/hub') .withCapabilities(options) .build(); await driver.get("http://localhost:8080"); const title = await driver.getTitle(); console.log(title); await driver.quit(); } runCodePipelineTest(); ``` Leave out the `tunnel-identifier` capability when you are testing a public website without a tunnel. ## Grouping Builds By passing the `build` capability, all sessions started by one CodeBuild run are grouped into a single TestingBot [build](https://testingbot.com/support/other/status-badges#builds), which gives you one combined pass or fail result for the pipeline execution. CodeBuild exposes several variables you can use for this: | Variable | Description | | --- | --- | | `CODEBUILD_BUILD_NUMBER` | Incrementing build number of the CodeBuild project. Short and readable, ideal as a build name. | | `CODEBUILD_BUILD_ID` | Unique id of the build, in the form `project-name:uuid`. | | `CODEBUILD_RESOLVED_SOURCE_VERSION` | The commit that triggered the pipeline. Useful as part of the test name. | | `CODEBUILD_WEBHOOK_HEAD_REF` | The branch of the commit, when the build was started by a webhook. | You can report the outcome of your build back to TestingBot with the [TestingBot REST API](https://testingbot.com/support/api), or fetch the status of a build to gate a deploy stage on the test results. ## Verify the Results In the AWS console, open **CodePipeline** , select your pipeline and choose **Details** on the build stage to follow the CodeBuild log. At the same time, your sessions appear in the [TestingBot member area](https://testingbot.com/members/tests), with video, screenshots, WebDriver logs and network traffic for every test. ## Troubleshooting - **Authentication failed** : the build could not read your credentials. Check that the CodeBuild service role is allowed to read the secret or parameter, and that the key and secret are not swapped. - **The tunnel never becomes ready** : make sure a Java runtime is present in the `install` phase (`java: corretto17`) and inspect `/tmp/tunnel.log`. If your VPC blocks outbound traffic, allow port 443 to `hub.testingbot.com` and `*.testingbot.com`. - **Tests time out on a local URL** : verify that the `tunnel-identifier` capability matches the `--tunnel-identifier` value passed to the tunnel, and that your local server is already listening when the tests start. - **The build hangs after the tests** : the tunnel is still running in the background. Stop it in the `post_build` phase as shown above. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/ci-cd/azure # Azure Devops (VSTS) Azure Devops (formerly Team Foundation Server and Visual Studio Team System) is a CI service owned by Microsoft. TestingBot has a [Marketplace Extension](https://marketplace.visualstudio.com/items?itemName=testingbot.comingbot-tasks) that offers some features to better integrate TestingBot with Azure Devops: - Set TestingBot environment variables (`TB_KEY` and `TB_SECRET`) to be used by your tests. - Upload a mobile app (`.apk`, `.aab`, `.ipa` or `.zip`) to [TestingBot Storage](https://testingbot.com/support/app-automate/appium/upload-files) for your Appium tests. - Embed overview of your tests in Azure Devops. - Start/Stop [TestingBot Tunnel](https://testingbot.com/support/tunnel) for specific builds. ## Installing Azure Pipelines extension 1. Make sure you are logged in to Azure Devops. Then please go to the [Visual Studio Marketplace](https://marketplace.visualstudio.com/items?itemName=testingbot.comingbot-tasks) and download the TestingBot extension. 2. Click on **Get it free** to install the extension. 3. Using the **Select an Azure DevOps organization** drop-down menu, choose the organization and click **install**. ## Configure the pipeline Now that the TestingBot extension has been installed, you'll need to configure the **Service Connection**. - Go to **Project Settings** and select **Service connections**. - Click **New Service Connection**. You can now add the **TestingBot credentials**. ![service connection](https://testingbot.com/assets/support/azure/serviceconnection-28f4f9d1e45db5b5e792de642feec247ddbaab1b80b4e26da2d3118cd8c27d60.png) ![new service connection](https://testingbot.com/assets/support/azure/newserviceconnection-fd4a7a4304e4de7459212127db1320f8c6a2561a1768113d7cb4ddc7bbd6c2de.png) - Add a **Service connection name** and click **Save**. Now that we've set up the Service Connection, we can go ahead and add the custom TestingBot task to the `azure-pipelines.yml` file. To do this, you can edit the file in your favorite editor, or through the YAML editor on Azure Devops. ### TestingBot Task In the `steps` block, you can now add the TBMain task, either via the Task menu in the Azure Devops page or by editing the file: ```yaml - task: TBMain@0 inputs: connectedServiceName: '{your service-connection-name}' ``` ![TestingBot task](https://testingbot.com/assets/support/azure/task-9cbd7e143fe2522166c111a23b86ce72574b371fb3d686e89195a5aa9a26774e.png) ![TestingBot credentials](https://testingbot.com/assets/support/azure/credentials-01bcbdd8a1459dfde3b813b282f58e2296e2a420d97cac3cc54ef259f1489879.png) ## Uploading a Mobile App To run Appium tests against a native app, you can upload your app build (`.apk`, `.aab`, `.ipa` or `.zip`) to [TestingBot Storage](https://testingbot.com/support/app-automate/appium/upload-files) directly from your pipeline with the **Upload App to TestingBot Storage** task. Add the `TBUploadApp` task before your test step and point it at the app you built earlier in the pipeline: ```yaml - task: TBUploadApp@0 inputs: connectedServiceName: '{your service-connection-name}' appFile: '$(Build.ArtifactStagingDirectory)/app.apk' ``` The task uploads the file and exports the returned `tb://` app URL as a pipeline variable (`TB_APP_URL` by default). Pass that value as the Appium `app` capability in your test: ```java MutableCapabilities capabilities = new MutableCapabilities(); capabilities.setCapability("platformName", "Android"); capabilities.setCapability("appium:deviceName", "Galaxy S23"); capabilities.setCapability("appium:automationName", "UiAutomator2"); // the app uploaded by the TBUploadApp task capabilities.setCapability("appium:app", System.getenv("TB_APP_URL")); capabilities.setCapability("build", System.getenv("TB_BUILD_NAME")); ``` Enable **Overwrite an existing app** and provide the app's `tb://` URL (or appkey) in `appId` to keep the same app URL across builds — handy when you always want to test the latest build without changing your test's capability. To output the app URL under a different variable name, set the `outputVariable` input. ## Viewing Test Results The extension allows you to view test results from Azure. To enable this, make sure you followed the previous steps and modify your existing test or test suite to send an extra capability to TestingBot: `build`. The extension makes the following environment variables available for your test cases: - `TB_KEY` - `TB_SECRET` - `TB_BUILD_NAME` The `TB_BUILD_NAME` environment variable needs to be passed as a `build` capability, for example: ```java ChromeOptions options = new ChromeOptions(); options.setCapability("browserVersion", "latest"); options.setCapability("platformName", "WIN11"); options.setCapability("name", methodName); // this capability below will make sure the reporting works options.setCapability("build", System.getenv("TB_BUILD_NAME")); webDriver.set(new RemoteWebDriver( new URL("https://" + System.getenv("TB_KEY") + ":" + System.getenv("TB_SECRET") + "@hub.testingbot.com/wd/hub"), options)); ``` It is **important** to pass the `TB_BUILD_NAME` environment variable as a `build` capability for the reporting to work. Once the build is complete, you will see a **TestingBot** tab. This shows an overview of the tests for this build: each test's name, status ( **passed** , **failed** or **unknown** when a test never reported its result), the browser/OS environment and the duration. Click a test to watch its video recording and inspect the logs and screenshots without leaving Azure DevOps. ![TestingBot embed test results](https://testingbot.com/assets/support/azure/embed-929bd7a5ec91895057a53ad7d9b109987a5cb18b5b389b846099e785cf0c6570.png) ## Enable TestingBot Tunnel [TestingBot Tunnel](https://testingbot.com/support/tunnel) allows you to run tests against web applications hosted on private, staging or development machines. To enable the TestingBot Tunnel in Azure, make sure the **TestingBot Tunnel checkbox** is checked. ![TestingBot enable tunnel](https://testingbot.com/assets/support/azure/enabletunnel-7d4aa19e75cc9b2a8fd0d25ed765c6ddef20697fabc58b7f891885753df22432.png) To use the Tunnel with your test, make sure to edit your test to use the Tunnel IP (`localhost:4445`): ```java ChromeOptions options = new ChromeOptions(); options.setCapability("browserVersion", "latest"); options.setCapability("platformName", "WIN11"); options.setCapability("name", methodName); // this capability below will make sure the reporting works options.setCapability("build", System.getenv("TB_BUILD_NAME")); webDriver.set(new RemoteWebDriver( new URL("http://" + System.getenv("TB_KEY") + ":" + System.getenv("TB_SECRET") + "@localhost:4445/wd/hub"), options)); ``` ### Stop Tunnel It's important to close the tunnel at the end of your test. To do this, please add the **Stop TestingBot Tunnel task** in your `azure-pipelines.yml` file. Make sure to add the `TBStopTunnel@0` after your other tasks. ![TestingBot stop tunnel](https://testingbot.com/assets/support/azure/stoptunnel-4bf8fe352a656bc1c06d20e08c6ee3bcfdf8a41bc7caede004d622fc3758ad6f.png) ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/ci-cd/bamboo # Bamboo Plugin TestingBot has created a plugin to be used with Atlassian's Bamboo CI: [TestingBot Plugin for Bamboo](https://marketplace.atlassian.com/apps/1221640/testingbot-plugin-for-bamboo) This plugin integrates TestingBot in Bamboo, features include: - Configure builds to use TestingBot: add environment variables with your TestingBot credentials - See a video, logs and screenshots of each test in Bamboo - Optionally start/stop [TestingBot Tunnel](https://testingbot.com/support/tunnel) for each build. ## Installation via Website To get started, go to the [Atlassian MarketPlace](https://marketplace.atlassian.com/apps/1221640/testingbot-plugin-for-bamboo), and click the **Get it now** button for the **TestingBot Plugin for Bamboo**. This will download a `.jar` file that you can use. ## Installation from inside Bamboo - Log into your Bamboo instance as an admin. - Click the admin dropdown and choose **Add-ons**. - Click **Find new apps** or **Find new add-ons** from the left-hand side of the page. - Locate **TestingBot Plugin for Bamboo** via search. - Click **Install** to download and install your app. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/ci-cd/bitbucket # Bitbucket Pipelines WebDriver Testing [Bitbucket Pipelines](https://bitbucket.org/product/features/pipelines) is the CI/CD service built into Bitbucket Cloud. Every push runs a set of steps in a Docker container, configured with a single `bitbucket-pipelines.yml` file in the root of your repository. Running your Selenium or Appium tests on the TestingBot browser and device cloud from a pipeline takes two things: make your TestingBot key and secret available to the build as repository variables, and point your tests at `https://hub.testingbot.com/wd/hub`. Nothing needs to be installed on the build container, because the browsers and devices run on our side. If your website is not publicly reachable, add a [TestingBot Tunnel](https://testingbot.com/support/tunnel) to the pipeline. This page covers both cases, together with grouping builds, parallel steps and reporting results back into the Bitbucket pull request. ## Prerequisites - A Bitbucket Cloud repository with Pipelines enabled ( **Repository settings** → **Pipelines** → **Settings** → **Enable Pipelines** ). - A `bitbucket-pipelines.yml` file in the root of that repository. - An existing Selenium or Appium test suite. - A TestingBot key and secret, available in the [TestingBot member area](https://testingbot.com/members/user/edit). ## Credentials Your tests read the TestingBot key and secret from the `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` environment variables. Add them in **Repository settings** → **Pipelines** → **Repository variables**. Tick the **Secured** checkbox for the secret: Bitbucket then stores the value encrypted and replaces it with the variable name everywhere it would otherwise appear in the build log. Bitbucket offers three scopes, which are resolved in this order: | Scope | When to use it | | --- | --- | | Workspace variables | One TestingBot account shared by every repository in the workspace. Managed by workspace admins. | | Repository variables | The usual choice. Overrides the workspace value, so one repository can use a different TestingBot account. | | Deployment variables | Different credentials per environment, for example a sub account for staging runs. Only applied to steps that declare a `deployment:`. | Never commit your TestingBot secret to `bitbucket-pipelines.yml`. Anyone with read access to the repository, including forks of a public repository, would be able to use your account. ## Configuration Create a `bitbucket-pipelines.yml` in the root of your repository. Pick an image that matches your test suite, install the dependencies and run the tests: [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [Ruby](https://testingbot.com#) ```yaml image: node:20 pipelines: default: - step: name: Selenium tests on TestingBot caches: - node script: - npm ci - npm run test:e2e ``` ```yaml image: python:3.12 pipelines: default: - step: name: Selenium tests on TestingBot caches: - pip script: - pip install -r requirements.txt - pytest --junitxml=test-results/results.xml ``` ```yaml image: maven:3.9-eclipse-temurin-21 pipelines: default: - step: name: Selenium tests on TestingBot caches: - maven script: - mvn -B clean test ``` ```yaml image: ruby:3.4 pipelines: default: - step: name: Selenium tests on TestingBot caches: - bundler script: - bundle install - bundle exec rspec ``` The `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` repository variables are injected automatically, so there is nothing to add to the `script` for authentication. The `caches` keyword reuses the dependency directory between builds, which usually saves more build minutes than anything else on this page. Bitbucket ships predefined caches for `node`, `pip`, `maven`, `gradle`, `bundler`, `composer` and others. ## Example Test Your test reads the credentials from the environment and sends the TestingBot capabilities in the `tb:options` object: [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [Ruby](https://testingbot.com#) ```javascript const webdriver = require('selenium-webdriver'); const chrome = require('selenium-webdriver/chrome'); const build = `bitbucket-${process.env.BITBUCKET_BUILD_NUMBER}`; async function runBitbucketTest () { let options = new chrome.Options(); options.set('platformName', 'WIN11'); options.set('browserVersion', 'latest'); options.set('tb:options', { 'key': process.env.TESTINGBOT_KEY, 'secret': process.env.TESTINGBOT_SECRET, 'name': 'Bitbucket Pipelines Test', 'build': build, 'tunnel-identifier': build }); let driver = new webdriver.Builder() .usingServer('https://hub.testingbot.com/wd/hub') .withCapabilities(options) .build(); await driver.get('http://localhost:8080'); console.log(await driver.getTitle()); await driver.quit(); } runBitbucketTest(); ``` ```python import os from selenium import webdriver from selenium.webdriver.chrome.options import Options build = "bitbucket-%s" % os.environ.get('BITBUCKET_BUILD_NUMBER') options = Options() options.set_capability('platformName', 'WIN11') options.set_capability('browserVersion', 'latest') options.set_capability('tb:options', { 'key': os.environ['TESTINGBOT_KEY'], 'secret': os.environ['TESTINGBOT_SECRET'], 'name': 'Bitbucket Pipelines Test', 'build': build, 'tunnel-identifier': build }) driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options ) driver.get("http://localhost:8080") print(driver.title) driver.quit() ``` ```java import org.openqa.selenium.WebDriver; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URL; import java.util.HashMap; import java.util.Map; public class BitbucketTest { public static final String KEY = System.getenv("TESTINGBOT_KEY"); public static final String SECRET = System.getenv("TESTINGBOT_SECRET"); public static final String URL = "https://hub.testingbot.com/wd/hub"; public static void main(String[] args) throws Exception { String build = "bitbucket-" + System.getenv("BITBUCKET_BUILD_NUMBER"); ChromeOptions options = new ChromeOptions(); options.setCapability("browserVersion", "latest"); options.setCapability("platformName", "WIN11"); Map tbOptions = new HashMap<>(); tbOptions.put("key", KEY); tbOptions.put("secret", SECRET); tbOptions.put("name", "Bitbucket Pipelines Test"); tbOptions.put("build", build); tbOptions.put("tunnel-identifier", build); options.setCapability("tb:options", tbOptions); WebDriver driver = new RemoteWebDriver(new URL(URL), options); driver.get("http://localhost:8080"); System.out.println(driver.getTitle()); driver.quit(); } } ``` ```ruby require 'selenium-webdriver' build = "bitbucket-#{ENV['BITBUCKET_BUILD_NUMBER']}" options = Selenium::WebDriver::Chrome::Options.new options.add_option('platformName', 'WIN11') options.add_option('browserVersion', 'latest') options.add_option('tb:options', { 'key' => ENV['TESTINGBOT_KEY'], 'secret' => ENV['TESTINGBOT_SECRET'], 'name' => 'Bitbucket Pipelines Test', 'build' => build, 'tunnel-identifier' => build }) driver = Selenium::WebDriver.for( :remote, url: "https://hub.testingbot.com/wd/hub", options: options) driver.navigate.to "http://localhost:8080" puts driver.title driver.quit ``` Leave out the `tunnel-identifier` capability when you are testing a publicly reachable website without a tunnel. ## Grouping Builds By passing the `build` capability, all sessions started by one pipeline run are grouped into a single TestingBot [build](https://testingbot.com/support/other/status-badges#builds), which gives you one combined pass or fail result per commit instead of a long list of unrelated tests. Bitbucket exposes a number of default variables you can use to name your builds and tests: | Variable | Description | | --- | --- | | `BITBUCKET_BUILD_NUMBER` | Incrementing build number. Short and readable, the best default for the `build` capability. | | `BITBUCKET_COMMIT` | Hash of the commit that started the pipeline. Useful in the test `name`. | | `BITBUCKET_BRANCH` | The source branch. Not set for tag builds. | | `BITBUCKET_PR_ID` | The pull request id, only set for pull request pipelines. | | `BITBUCKET_PARALLEL_STEP` | Zero based index of the step inside a parallel group. Handy as a unique tunnel identifier. | | `BITBUCKET_EXIT_CODE` | Result of the step, only available inside `after-script`. `0` means success. | A readable convention is to use the build number for the `build` capability and the branch plus commit for the test `name`, so you can trace a failing session straight back to a commit. ## Privately Hosted Websites If the website under test is not publicly reachable, a staging environment behind a firewall or a server started by the build itself, run a [TestingBot Tunnel](https://testingbot.com/support/tunnel) in the pipeline. There are two ways to do this in Bitbucket Pipelines. ### Option 1: start the tunnel in the step The tunnel is a Java application, so use an image that has a JRE available or install one first. Start the tunnel in the background, wait for its ready file, then run your tests: ```yaml image: node:20 pipelines: default: - step: name: Selenium tests on TestingBot caches: - node script: - apt-get update && apt-get install -y --no-install-recommends default-jre-headless unzip - npm ci - curl -sS -O https://testingbot.com/downloads/testingbot-tunnel.zip - unzip -o -q testingbot-tunnel.zip - | java -jar testingbot-tunnel.jar $TESTINGBOT_KEY $TESTINGBOT_SECRET \ --tunnel-identifier bitbucket-$BITBUCKET_BUILD_NUMBER \ --readyfile tunnel.ready \ --logfile tunnel.log & - timeout 180 bash -c 'until [-f tunnel.ready]; do sleep 1; done' - npm run start:staging & - npm run test:e2e after-script: - pkill -f testingbot-tunnel || true artifacts: - tunnel.log ``` A few things worth pointing out: - The `--readyfile` flag touches a file as soon as the tunnel is connected. Waiting for that file is more reliable than a fixed `sleep`, because your tests never start before the tunnel is up. Keep both the ready file and the log inside the clone directory, otherwise Bitbucket cannot pick the log up as an artifact. - The `--tunnel-identifier` flag names the tunnel, so concurrent pipelines each get their own. Pass the same value as the `tunnel-identifier` capability in your tests. - `after-script` runs whether the step passed or failed, which makes it the right place to stop the tunnel. Without it the step keeps running until the pipeline times out. - The `testingbot-tunnel.zip` download always contains the latest tunnel version, which is why the example uses a wildcard for the jar name. ### Option 2: run the tunnel as a service container Bitbucket service containers share their network adapter with the build container, so a tunnel started as a service is reachable on `localhost` and can in turn reach any server your step starts. This keeps your `script` free of tunnel plumbing and works with an image that has no Java, using the official `testingbot/tunnel` image from Docker Hub: ```yaml image: node:20 definitions: services: testingbot-tunnel: image: testingbot/tunnel memory: 1024 variables: TESTINGBOT_KEY: $TESTINGBOT_KEY TESTINGBOT_SECRET: $TESTINGBOT_SECRET pipelines: default: - step: name: Selenium tests on TestingBot services: - testingbot-tunnel caches: - node script: - npm ci - npm run start:staging & - timeout 180 bash -c 'until (echo > /dev/tcp/127.0.0.1/4445) 2>/dev/null; do sleep 2; done' - npm run test:e2e ``` With this setup you do not need a tunnel identifier. Point your tests at the Selenium relay the tunnel exposes on `http://localhost:4445/wd/hub` instead of at `https://hub.testingbot.com/wd/hub`: the relay forwards your sessions to TestingBot and routes the traffic back through the tunnel. See the [tunnel command line options](https://testingbot.com/support/tunnel/commandline) for all available flags. Bitbucket does not wait for a service to become healthy before it starts your script, so keep the polling loop above. A step can use at most 5 services, and all services together share the memory budget of the step (3072 MB on a regular step, 7128 MB on a `size: 2x` step). ## Parallel Steps Cross browser suites are the obvious candidate for Bitbucket's `parallel` steps: each step runs on its own build container at the same time, so the wall clock time of the pipeline is that of the slowest browser rather than the sum of all of them. ```yaml pipelines: default: - parallel: - step: name: Chrome on Windows 11 script: - BROWSER=chrome PLATFORM=WIN11 npm run test:e2e - step: name: Safari on macOS script: - BROWSER=safari PLATFORM=SONOMA npm run test:e2e - step: name: Firefox on Windows 11 script: - BROWSER=firefox PLATFORM=WIN11 npm run test:e2e ``` Use the same `build` capability in every step, so the three steps still show up as one TestingBot build. If your steps each start a tunnel, give them a unique identifier using `bitbucket-$BITBUCKET_BUILD_NUMBER-$BITBUCKET_PARALLEL_STEP`. The number of steps you can usefully run at once is limited by the concurrency of your TestingBot plan. Extra sessions are queued rather than rejected, so a pipeline with more parallel steps than available slots still passes, it just takes longer. You can see your current concurrency in the [member area](https://testingbot.com/members). ## Test Reports Bitbucket parses JUnit style XML and shows failing tests on a **Tests** tab next to the build log. It scans these directories automatically, up to three levels deep: - `./ **/test-results/** /*.xml` - `./ **/test-reports/** /*.xml` - `./ **/surefire-reports/** /*.xml` - `./ **/failsafe-reports/** /*.xml` - `./ **/TestResults/** /*.xml` If your test runner writes its report somewhere else, declare it explicitly: ```yaml - step: name: Selenium tests on TestingBot script: - npm run test:e2e artifacts: upload: - name: "e2e test reports" type: "test-reports" paths: - "reports/junit/*.xml" ``` This gives you the failure summary inside Bitbucket, while the video, screenshots, WebDriver log and network traffic of every session stay available on TestingBot. ## Pull Request Reports Bitbucket [Code Insights](https://support.atlassian.com/bitbucket-cloud/docs/code-insights/) lets a pipeline attach a report to a commit, which then shows up in the sidebar of every pull request containing that commit. It is a good way to surface a link to your TestingBot build without leaving the review flow. The quickest way is our **TestingBot Report** pipe, which reads the TestingBot build and writes the report for you. Put it in `after-script`, so it also runs when your tests fail: ```yaml - step: name: Selenium tests on TestingBot script: - npm run test:e2e after-script: - pipe: testingbot/testingbot-report-pipe:0.1.0 variables: TESTINGBOT_KEY: $TESTINGBOT_KEY TESTINGBOT_SECRET: $TESTINGBOT_SECRET ``` The pipe looks up the build named `bitbucket-$BITBUCKET_BUILD_NUMBER`, which is what the `build` capability in the example test above already sets. It publishes the pass and fail counts plus the total duration, and adds one annotation per failing test, each linking to the session on TestingBot. Set `FAIL_ON_FAILED_TESTS: 'true'` if you want the pipe itself to fail the step. ### Without the pipe You can also write the report yourself. Inside a pipeline you do not need any credentials for this: Pipelines runs an authenticating proxy on `localhost:29418` that signs the request for you. ```yaml - step: name: Selenium tests on TestingBot script: - npm run test:e2e after-script: - | if ["$BITBUCKET_EXIT_CODE" = "0"]; then RESULT=PASSED; else RESULT=FAILED; fi cat > report.json < ENV['TESTINGBOT_KEY'], 'secret' => ENV['TESTINGBOT_SECRET'], 'name' => 'Buildkite Test', 'build' => ENV['TESTINGBOT_BUILD'], 'tunnel-identifier' => ENV['TESTINGBOT_TUNNEL_IDENTIFIER'] }) driver = Selenium::WebDriver.for( :remote, url: "https://hub.testingbot.com/wd/hub", options: options) driver.navigate.to "http://localhost:8080" puts driver.title driver.quit ``` ```python import os from selenium import webdriver from selenium.webdriver.chrome.options import Options options = Options() options.set_capability('platformName', 'WIN11') options.set_capability('browserVersion', 'latest') options.set_capability('tb:options', { 'key': os.environ['TESTINGBOT_KEY'], 'secret': os.environ['TESTINGBOT_SECRET'], 'name': 'Buildkite Test', 'build': os.environ.get('TESTINGBOT_BUILD'), 'tunnel-identifier': os.environ.get('TESTINGBOT_TUNNEL_IDENTIFIER') }) driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options ) driver.get("http://localhost:8080") print(driver.title) driver.quit() ``` ```java import org.openqa.selenium.WebDriver; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URL; import java.util.HashMap; import java.util.Map; public class BuildkiteTest { public static final String KEY = System.getenv("TESTINGBOT_KEY"); public static final String SECRET = System.getenv("TESTINGBOT_SECRET"); public static final String URL = "https://hub.testingbot.com/wd/hub"; public static void main(String[] args) throws Exception { ChromeOptions options = new ChromeOptions(); options.setCapability("browserVersion", "latest"); options.setCapability("platformName", "WIN11"); Map tbOptions = new HashMap<>(); tbOptions.put("key", KEY); tbOptions.put("secret", SECRET); tbOptions.put("name", "Buildkite Test"); tbOptions.put("build", System.getenv("TESTINGBOT_BUILD")); tbOptions.put("tunnel-identifier", System.getenv("TESTINGBOT_TUNNEL_IDENTIFIER")); options.setCapability("tb:options", tbOptions); WebDriver driver = new RemoteWebDriver(new URL(URL), options); driver.get("http://localhost:8080"); System.out.println(driver.getTitle()); driver.quit(); } } ``` ```javascript const webdriver = require('selenium-webdriver'); const chrome = require('selenium-webdriver/chrome'); async function runBuildkiteTest () { let options = new chrome.Options(); options.set('platformName', 'WIN11'); options.set('browserVersion', 'latest'); options.set('tb:options', { 'key': process.env.TESTINGBOT_KEY, 'secret': process.env.TESTINGBOT_SECRET, 'name': 'Buildkite Test', 'build': process.env.TESTINGBOT_BUILD, 'tunnel-identifier': process.env.TESTINGBOT_TUNNEL_IDENTIFIER }); let driver = new webdriver.Builder() .usingServer('https://hub.testingbot.com/wd/hub') .withCapabilities(options) .build(); await driver.get("http://localhost:8080"); const title = await driver.getTitle(); console.log(title); await driver.quit(); } runBuildkiteTest(); ``` ## Reporting Sessions to the Plugin For status updates and annotations, the plugin needs to know which TestingBot sessions belong to the build step. There are two mechanisms, checked in this order: ### 1. Sessions file Append each WebDriver session id to the file named by `$TESTINGBOT_SESSIONS_FILE` (default `testingbot-sessions.txt` in the checkout), one per line, optionally followed by a per-test status and name: ```bash [passed|failed] [test name] ``` For example in a WebdriverIO `afterSession` hook: ```javascript afterSession: async function (config, capabilities, specs) { const status = global.testFailed ? 'failed' : 'passed'; fs.appendFileSync(process.env.TESTINGBOT_SESSIONS_FILE, `${browser.sessionId} ${status} ${specs[0]}\n`); } ``` Per-line statuses override the step outcome, giving you accurate per-test results. This mechanism is recommended when multiple parallel Buildkite jobs share one build. ### 2. Build capability Alternatively, set the `build` capability of your tests to `$TESTINGBOT_BUILD`, as shown in the [example above](https://testingbot.com#example). The plugin then finds the sessions through the [TestingBot API](https://testingbot.com/support/api), without any code changes to your tests. ## Test Statuses After your command finishes, the plugin marks every reported session as passed or failed on TestingBot: - If the build step succeeded, sessions are marked as **passed**. - If the build step failed, sessions are marked as **failed**. - A per-line `passed`/`failed` status in the sessions file always takes precedence. Each test's status message links back to the Buildkite build, so you can navigate from a test on TestingBot straight to the build that ran it. Set `update-status: false` to disable this behavior. ## Build Annotations The plugin annotates the Buildkite build with a results table: pass/fail per test, platform, duration, and links to the report and video of each test on TestingBot. Failed tests get a collapsible detail section, with a screenshot of the failure uploaded as a Buildkite artifact. By default, links in the annotation use TestingBot [share URLs](https://testingbot.com/support/other/sharing): anyone who can see the Buildkite build page can open the test report and video without a TestingBot login. The share hash only grants view access to that specific test. Set `share-links: false` to use links that require a TestingBot login instead. ![A TestingBot test opened through a share link](https://testingbot.com/assets/support/buildkite/testingbot-share-page-c8fe9ed5bea48ace5537d473d32c086356254e02df12317e258d1e16c40f16e1.webp) ## Configuration The TestingBot Buildkite Plugin accepts the following configuration options: | Option | Description | | --- | --- | | `tunnel` | Start a TestingBot Tunnel before the command and stop it after. Default is `true`. | | `tunnel-identifier` | Identifier for the tunnel connection. `%job-id%` is replaced with the Buildkite job id, recommended for parallel jobs. Exported as `$TESTINGBOT_TUNNEL_IDENTIFIER`. | | `tunnel-args` | Extra flags passed verbatim to the tunnel, see the [command line options](https://testingbot.com/support/tunnel/commandline). | | `tunnel-ready-timeout` | Seconds to wait for the tunnel to become ready before failing the step. Default is `120`. | | `tunnel-download-url` | Override the tunnel download URL, for example an internal mirror. | | `annotate` | Create a build annotation with the test results. Default is `true`. Rich annotations require `jq` on the agent. | | `share-links` | Use no-login [share URLs](https://testingbot.com/support/other/sharing) in the annotation. Default is `true`. | | `thumbnails` | Upload failure screenshots as Buildkite artifacts and embed them in the annotation. Default is `true`. | | `update-status` | Update TestingBot test statuses based on the step outcome. Default is `true`. | | `sessions-file` | Path, relative to the checkout, where tests write session ids. Default is `testingbot-sessions.txt`. Exported as `$TESTINGBOT_SESSIONS_FILE`. | | `build-identifier` | TestingBot build identifier, exported as `$TESTINGBOT_BUILD`. Default is `pipeline-slug-build-number`. | | `api-key-env` | Name of the environment variable holding your TestingBot key. Default is `TESTINGBOT_KEY`. | | `api-secret-env` | Name of the environment variable holding your TestingBot secret. Default is `TESTINGBOT_SECRET`. | | `strict` | Fail the step when status updates or annotations fail. Default is `false` (warn only). | ## Artifacts The plugin uploads the tunnel log as a Buildkite artifact for every job, so you can inspect what happened with the TestingBot Tunnel. When a test fails, its failure screenshot is uploaded as an artifact as well and embedded in the [build annotation](https://testingbot.com#annotations). ## More Information The plugin is open source and listed in the [Buildkite plugins directory](https://buildkite.com/resources/plugins/testingbot/testingbot-buildkite-plugin/). More information, including framework examples and the full changelog, is available in the [TestingBot Buildkite Plugin repository](https://github.com/testingbot/testingbot-buildkite-plugin). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/ci-cd/circleci # CircleCI Automated Testing CircleCI is a continuous integration service in the cloud. With the official [TestingBot orb](https://circleci.com/developer/orbs/orb/testingbot/testingbot) you can run Selenium, Appium, Playwright and Cypress tests on the TestingBot browser and real device cloud straight from your CircleCI pipeline, and expose an application that only runs inside your CircleCI job to our cloud via the [TestingBot Tunnel](https://testingbot.com/support/tunnel). Once your tests are up and running you can display a TestingBot [status badge](https://testingbot.com/support/other/status-badges) showcasing the status of your test build. ## Get CircleCI up and running Sign up at [CircleCI](https://circleci.com) and connect your project with CircleCI. We recommend setting up [environment variables](https://circleci.com/docs/guides/security/set-environment-variable/) on the project settings page of CircleCI. You can set up the `TB_KEY` and `TB_SECRET` variables there, so that they are not publicly visible. Both variables can also be stored in a CircleCI [context](https://circleci.com/docs/guides/security/contexts/), so several projects share the same TestingBot credentials. ## The TestingBot CircleCI Orb The TestingBot orb is published in the CircleCI registry as `testingbot/testingbot`. Its source lives on [GitHub](https://github.com/testingbot/circleci-orb). The orb takes care of the [TestingBot Tunnel](https://testingbot.com/support/tunnel) for you: it downloads the tunnel, starts it, waits until it is ready to accept traffic and shuts it down again when the job finishes. That makes it possible to test a web application that only exists inside the CircleCI job, such as a dev server on `localhost`, a staging environment, or any host behind your firewall. The orb provides: - A `with_tunnel` job that wraps your test steps and handles setup and teardown. - Three commands, `install_tunnel`, `start_tunnel` and `stop_tunnel`, for when you want to control the tunnel inside your own job. - A `default` executor based on `cimg/openjdk`, which ships the Java runtime the tunnel needs. **Java is required.** The TestingBot Tunnel is a Java application and needs Java 17 or newer (tunnel 5.0 and later; 4.x ran on Java 11). The orb's `default` executor already provides it. If you bring your own executor, make sure it has a JRE installed, for example the `cimg/openjdk:17.0` image. ## Quick start: the with\_tunnel job The fastest way to get started is the `with_tunnel` job. Pass it the steps you want to run, and the orb installs and starts the tunnel, runs your steps and always stops the tunnel afterwards, even when your tests fail. **Example `.circleci/config.yml` file:** ```yaml version: 2.1 orbs: testingbot: testingbot/testingbot@1.0.0 workflows: test: jobs: - testingbot/with_tunnel: tunnel_identifier: circleci-<< pipeline.number >> steps: - run: name: Run cross-browser tests on TestingBot command: npm run test:e2e ``` Your credentials are read from the `TB_KEY` and `TB_SECRET` environment variables you configured above. If you store them under different names, point the `key` and `secret` parameters at those variable names. ## Using the commands in your own job When you need more control, for example because you want to build the application first or use a specific Docker image, use the individual commands instead: ```yaml version: 2.1 orbs: testingbot: testingbot/testingbot@1.0.0 jobs: e2e-tests: executor: testingbot/default steps: - checkout - testingbot/install_tunnel - testingbot/start_tunnel: tunnel_identifier: my-tunnel - run: name: Run Selenium tests through the tunnel command: npm run test:e2e - testingbot/stop_tunnel workflows: test: jobs: - e2e-tests ``` `stop_tunnel` runs with `when: always`, so the tunnel is cleaned up even when an earlier step failed. **Building a tunnel identifier at runtime.** The `tunnel_identifier` parameter is a literal string: shell variables inside it are not expanded. To compute an identifier in the job itself, leave the parameter empty and export `TB_TUNNEL_IDENTIFIER` via `$BASH_ENV` in an earlier step. CircleCI pipeline values such as `<< pipeline.number >>` are interpolated by CircleCI itself and can be used directly. ## Pointing your tests at the tunnel Once the tunnel is ready, it exposes a local Selenium relay on port `4445`. Point your tests at `http://localhost:4445/wd/hub` instead of `hub.testingbot.com`, and traffic is routed through the tunnel to the TestingBot cloud. Use the `se_port` parameter if you need a different port. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java import org.openqa.selenium.WebDriver; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URL; import java.util.HashMap; import java.util.Map; String url = "http://localhost:4445/wd/hub"; ChromeOptions options = new ChromeOptions(); options.setPlatformName(System.getenv("SELENIUM_PLATFORM")); options.setBrowserVersion(System.getenv("SELENIUM_BROWSER_VERSION")); Map tbOptions = new HashMap<>(); tbOptions.put("build", "CircleCI build " + System.getenv("CIRCLE_BUILD_NUM")); tbOptions.put("tunnelIdentifier", System.getenv("TB_TUNNEL_IDENTIFIER")); options.setCapability("tb:options", tbOptions); WebDriver driver = new RemoteWebDriver(new URL(url), options); ``` ```python import os from selenium import webdriver url = "http://localhost:4445/wd/hub" options = webdriver.ChromeOptions() options.set_capability('platformName', os.environ.get('SELENIUM_PLATFORM', 'WIN11')) options.set_capability('browserVersion', os.environ.get('SELENIUM_BROWSER_VERSION', 'latest')) options.set_capability('tb:options', { 'build': "CircleCI build {}".format(os.environ.get('CIRCLE_BUILD_NUM')), 'tunnelIdentifier': os.environ.get('TB_TUNNEL_IDENTIFIER', 'my-tunnel') }) driver = webdriver.Remote(command_executor=url, options=options) ``` ```javascript const { Builder } = require('selenium-webdriver'); const url = 'http://localhost:4445/wd/hub'; const driver = await new Builder() .usingServer(url) .withCapabilities({ browserName: process.env.SELENIUM_BROWSER || 'chrome', platformName: process.env.SELENIUM_PLATFORM || 'WIN11', browserVersion: process.env.SELENIUM_BROWSER_VERSION || 'latest', 'tb:options': { build: `CircleCI build ${process.env.CIRCLE_BUILD_NUM}`, tunnelIdentifier: process.env.TB_TUNNEL_IDENTIFIER || 'my-tunnel' } }) .build(); ``` ```ruby require 'selenium/webdriver' url = "http://localhost:4445/wd/hub" options = Selenium::WebDriver::Chrome::Options.new options.add_option('platformName', ENV['SELENIUM_PLATFORM'] || 'WIN11') options.add_option('browserVersion', ENV['SELENIUM_BROWSER_VERSION'] || 'latest') options.add_option('tb:options', { 'build' => "CircleCI build #{ENV['CIRCLE_BUILD_NUM']}", 'tunnelIdentifier' => ENV['TB_TUNNEL_IDENTIFIER'] || 'my-tunnel' }) browser = Selenium::WebDriver.for(:remote, url: url, options: options) ``` ```php use Facebook\WebDriver\Remote\RemoteWebDriver; use Facebook\WebDriver\Remote\DesiredCapabilities; $url = 'http://localhost:4445/wd/hub'; $capabilities = DesiredCapabilities::chrome(); $capabilities->setCapability('platformName', getenv('SELENIUM_PLATFORM') ?: 'WIN11'); $capabilities->setCapability('browserVersion', getenv('SELENIUM_BROWSER_VERSION') ?: 'latest'); $capabilities->setCapability('tb:options', [ 'build' => 'CircleCI build ' . getenv('CIRCLE_BUILD_NUM'), 'tunnelIdentifier' => getenv('TB_TUNNEL_IDENTIFIER') ?: 'my-tunnel' ]); $driver = RemoteWebDriver::create($url, $capabilities); ``` ```csharp using System; using System.Collections.Generic; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Remote; var url = "http://localhost:4445/wd/hub"; ChromeOptions options = new ChromeOptions(); options.PlatformName = Environment.GetEnvironmentVariable("SELENIUM_PLATFORM") ?? "WIN11"; options.BrowserVersion = Environment.GetEnvironmentVariable("SELENIUM_BROWSER_VERSION") ?? "latest"; options.AddAdditionalOption("tb:options", new Dictionary { ["build"] = $"CircleCI build {Environment.GetEnvironmentVariable("CIRCLE_BUILD_NUM")}", ["tunnelIdentifier"] = Environment.GetEnvironmentVariable("TB_TUNNEL_IDENTIFIER") ?? "my-tunnel" }); IWebDriver driver = new RemoteWebDriver(new Uri(url), options); ``` When several tunnels run in parallel, each one needs its own identifier. Pass the same value in the `tunnelIdentifier` capability so your test is routed through the correct tunnel. See [running multiple tunnels](https://testingbot.com/support/tunnel/multiple) for more detail. Setting the same `build` capability on every session in the workflow groups them into a single TestingBot build, which you can then show with a [status badge](https://testingbot.com/support/other/status-badges) or query from the [REST API](https://testingbot.com/support/api). ## Orb reference ### Job: with\_tunnel Runs your test steps with a TestingBot Tunnel active. | Parameter | Default | Description | | --- | --- | --- | | `steps` **Required** | - | Test steps to run while the tunnel is active. | | `key` | `TB_KEY` | Name of the environment variable holding your TestingBot key. | | `secret` | `TB_SECRET` | Name of the environment variable holding your TestingBot secret. | | `tunnel_identifier` | _empty_ | Optional named identifier for this tunnel. Required when running multiple tunnels in parallel. | | `se_port` | `4445` | Local port for the tunnel's Selenium relay. | | `ready_timeout` | `120` | Maximum time in seconds to wait for the tunnel to be ready. | | `extra_args` | _empty_ | Additional [command-line flags](https://testingbot.com/support/tunnel/commandline) passed verbatim to the tunnel. | | `checkout` | `true` | Whether to check out your code before starting the tunnel. | | `executor` | `default` | Executor to run the job on. Must provide Java 17 or newer. | ### Command: install\_tunnel Downloads and installs the TestingBot Tunnel jar. Skips the download when the jar is already present. | Parameter | Default | Description | | --- | --- | --- | | `install_dir` | `~/testingbot-tunnel` | Directory to install the tunnel jar into. | | `download_url` | `https://testingbot.com/downloads/testingbot-tunnel.zip` | URL of the TestingBot Tunnel zip archive to download. | ### Command: start\_tunnel Starts the tunnel in the background and waits until it is ready. Requires `install_tunnel` to have run first. | Parameter | Default | Description | | --- | --- | --- | | `key` | `TB_KEY` | Name of the environment variable holding your TestingBot key. | | `secret` | `TB_SECRET` | Name of the environment variable holding your TestingBot secret. | | `tunnel_identifier` | _empty_ | Optional named identifier for this tunnel. Pass the same value in your test capabilities. | | `se_port` | `4445` | Local port for the tunnel's Selenium relay. | | `ready_timeout` | `120` | Maximum time in seconds to wait for the tunnel to be ready. | | `logfile` | `/tmp/testingbot-tunnel.log` | Path of the tunnel log file, printed when the tunnel fails to start. | | `install_dir` | `~/testingbot-tunnel` | Directory where the tunnel jar was installed. | | `extra_args` | _empty_ | Additional flags passed verbatim to the tunnel jar, for example `--nobump --log-level debug`. | ### Command: stop\_tunnel Gracefully stops a tunnel started with `start_tunnel`. It takes no parameters and runs even when earlier steps failed. ### Executor: default A Docker executor based on `cimg/openjdk`, which provides the Java runtime the tunnel needs. Use the `tag` parameter (default `17.0`) to pick a different image tag: ```yaml jobs: e2e-tests: executor: name: testingbot/default tag: "21.0" ``` ## Testing without a tunnel If your application is publicly reachable, you do not need the tunnel at all. Point your tests directly at `hub.testingbot.com` and use whichever executor suits your test stack. Pick your language below for a `.circleci/config.yml` file and the matching test code. [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```yaml version: 2.1 jobs: test: parallelism: 3 # depends on how many concurrent sessions your TestingBot subscription has docker: - image: cimg/openjdk:17.0 environment: TB_BUILD: "build No. $CIRCLE_BUILD_NUM for CircleCI" SELENIUM_PLATFORM: WIN11 SELENIUM_BROWSER: chrome SELENIUM_BROWSER_VERSION: "latest" steps: - checkout - run: mvn -B test workflows: test: jobs: - test ``` ```yaml version: 2.1 jobs: test: parallelism: 3 # depends on how many concurrent sessions your TestingBot subscription has docker: - image: cimg/python:3.12 environment: TB_BUILD: "build No. $CIRCLE_BUILD_NUM for CircleCI" SELENIUM_PLATFORM: WIN11 SELENIUM_BROWSER: chrome SELENIUM_BROWSER_VERSION: "latest" steps: - checkout - run: pip install -r requirements.txt - run: pytest workflows: test: jobs: - test ``` ```yaml version: 2.1 jobs: test: parallelism: 3 # depends on how many concurrent sessions your TestingBot subscription has docker: - image: cimg/node:20.11 environment: TB_BUILD: "build No. $CIRCLE_BUILD_NUM for CircleCI" SELENIUM_PLATFORM: WIN11 SELENIUM_BROWSER: chrome SELENIUM_BROWSER_VERSION: "latest" steps: - checkout - run: npm ci - run: npm test workflows: test: jobs: - test ``` ```yaml version: 2.1 orbs: ruby: circleci/ruby@2.1.0 node: circleci/node@5.2.0 jobs: test: parallelism: 3 # depends on how many concurrent sessions your TestingBot subscription has docker: - image: cimg/ruby:3.2-node environment: TB_BUILD: "build No. $CIRCLE_BUILD_NUM for CircleCI" SELENIUM_PLATFORM: WIN11 SELENIUM_BROWSER: chrome SELENIUM_BROWSER_VERSION: "latest" steps: - checkout - ruby/install: version: '3.2' - ruby/install-deps - run: bundle exec cucumber workflows: test: jobs: - test ``` ```yaml version: 2.1 jobs: test: parallelism: 3 # depends on how many concurrent sessions your TestingBot subscription has docker: - image: cimg/php:8.3 environment: TB_BUILD: "build No. $CIRCLE_BUILD_NUM for CircleCI" SELENIUM_PLATFORM: WIN11 SELENIUM_BROWSER: chrome SELENIUM_BROWSER_VERSION: "latest" steps: - checkout - run: composer install --no-interaction - run: ./vendor/bin/phpunit workflows: test: jobs: - test ``` ```yaml version: 2.1 jobs: test: parallelism: 3 # depends on how many concurrent sessions your TestingBot subscription has docker: - image: mcr.microsoft.com/dotnet/sdk:8.0 environment: TB_BUILD: "build No. $CIRCLE_BUILD_NUM for CircleCI" SELENIUM_PLATFORM: WIN11 SELENIUM_BROWSER: chrome SELENIUM_BROWSER_VERSION: "latest" steps: - checkout - run: dotnet restore - run: dotnet test workflows: test: jobs: - test ``` You can now use the environment variables, defined above and on CircleCI, in your code: [Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [C#](https://testingbot.com#) ```java import org.openqa.selenium.WebDriver; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URL; import java.util.HashMap; import java.util.Map; String url = String.format("https://%s:%s@hub.testingbot.com/wd/hub", System.getenv("TB_KEY"), System.getenv("TB_SECRET")); ChromeOptions options = new ChromeOptions(); options.setPlatformName(System.getenv("SELENIUM_PLATFORM")); options.setBrowserVersion(System.getenv("SELENIUM_BROWSER_VERSION")); Map tbOptions = new HashMap<>(); tbOptions.put("build", System.getenv("TB_BUILD")); options.setCapability("tb:options", tbOptions); WebDriver driver = new RemoteWebDriver(new URL(url), options); ``` ```python import os from selenium import webdriver url = "https://{}:{}@hub.testingbot.com/wd/hub".format( os.environ['TB_KEY'], os.environ['TB_SECRET']) options = webdriver.ChromeOptions() options.set_capability('platformName', os.environ.get('SELENIUM_PLATFORM', 'WIN11')) options.set_capability('browserVersion', os.environ.get('SELENIUM_BROWSER_VERSION', 'latest')) options.set_capability('tb:options', { 'build': os.environ.get('TB_BUILD') }) driver = webdriver.Remote(command_executor=url, options=options) ``` ```javascript const { Builder } = require('selenium-webdriver'); const url = `https://${process.env.TB_KEY}:${process.env.TB_SECRET}@hub.testingbot.com/wd/hub`; const driver = await new Builder() .usingServer(url) .withCapabilities({ browserName: process.env.SELENIUM_BROWSER || 'chrome', platformName: process.env.SELENIUM_PLATFORM || 'WIN11', browserVersion: process.env.SELENIUM_BROWSER_VERSION || 'latest', 'tb:options': { build: process.env.TB_BUILD } }) .build(); ``` ```ruby require 'selenium/webdriver' url = "https://#{ENV['TB_KEY']}:#{ENV['TB_SECRET']}@hub.testingbot.com/wd/hub" options = Selenium::WebDriver::Chrome::Options.new options.add_option('platformName', ENV['SELENIUM_PLATFORM'] || 'WIN11') options.add_option('browserVersion', ENV['SELENIUM_BROWSER_VERSION'] || 'latest') options.add_option('tb:options', { 'build' => ENV['TB_BUILD'] }) browser = Selenium::WebDriver.for(:remote, url: url, options: options) Before do |scenario| @browser = browser end at_exit do browser.quit end ``` ```php use Facebook\WebDriver\Remote\RemoteWebDriver; use Facebook\WebDriver\Remote\DesiredCapabilities; $url = sprintf('https://%s:%s@hub.testingbot.com/wd/hub', getenv('TB_KEY'), getenv('TB_SECRET')); $capabilities = DesiredCapabilities::chrome(); $capabilities->setCapability('platformName', getenv('SELENIUM_PLATFORM') ?: 'WIN11'); $capabilities->setCapability('browserVersion', getenv('SELENIUM_BROWSER_VERSION') ?: 'latest'); $capabilities->setCapability('tb:options', [ 'build' => getenv('TB_BUILD') ]); $driver = RemoteWebDriver::create($url, $capabilities); ``` ```csharp using System; using System.Collections.Generic; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Remote; var url = $"https://{Environment.GetEnvironmentVariable("TB_KEY")}:{Environment.GetEnvironmentVariable("TB_SECRET")}@hub.testingbot.com/wd/hub"; ChromeOptions options = new ChromeOptions(); options.PlatformName = Environment.GetEnvironmentVariable("SELENIUM_PLATFORM") ?? "WIN11"; options.BrowserVersion = Environment.GetEnvironmentVariable("SELENIUM_BROWSER_VERSION") ?? "latest"; options.AddAdditionalOption("tb:options", new Dictionary { ["build"] = Environment.GetEnvironmentVariable("TB_BUILD") }); IWebDriver driver = new RemoteWebDriver(new Uri(url), options); ``` ## More information - [TestingBot orb in the CircleCI registry](https://circleci.com/developer/orbs/orb/testingbot/testingbot) - [Orb source code on GitHub](https://github.com/testingbot/circleci-orb) - [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) - [All tunnel command-line options](https://testingbot.com/support/tunnel/commandline) - [Status badges](https://testingbot.com/support/other/status-badges) ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/ci-cd/github-actions # TestingBot GitHub Action This guide will help you integrate the TestingBot GitHub Action workflow in your project. The TestingBot GitHub action provides an action to integrate the [TestingBot Tunnel](https://testingbot.com/support/tunnel) in your tests. The TestingBot Tunnel is used to route all test traffic between your web application and the TestingBot browser/device cloud. The GitHub action will also save the [test-artifacts](https://testingbot.com#artifacts) for you, so that you can later see the meta-data generated by the test. Running Maestro flows? Use the dedicated [TestingBot Maestro action](https://testingbot.com/support/app-automate/maestro/ci-cd/github-actions), which uploads the app, runs the flows on TestingBot devices and posts a [status check on the pull request](https://testingbot.com/support/app-automate/maestro/ci-cd/github-pr-checks). ## GitHub Secrets To get started, you'll need to configure 2 secrets in your GitHub repository. These 2 secrets are `TB_KEY` (your TestingBot Key) and `TB_SECRET` (TestingBot secret) which can be obtained from the TestingBot member area. In your GitHub repository, go to **Settings** and find the **Secrets** option in the left-hand pane. Add both the `TB_KEY` and `TB_SECRET` as **Repository Secrets** : ![GitHub Secrets](https://testingbot.com/assets/support/github/secrets-5e390bc9c169a9b3e4d247c00606334ada9202d731436d23d7f96c099d0b1829.webp) ## Set up a GitHub workflow Next, we'll create a new GitHub workflow. Create a workflow file `test-workflow.yml` in the `.github/workflows` directory of your GitHub repository. You can find out more about workflow files on the [GitHub documentation pages](https://docs.github.com/en/actions/learn-github-actions#creating-a-workflow-file). In your workflow file, you can indicate [when to run the GitHub Action](https://docs.github.com/en/actions/reference/events-that-trigger-workflows). We recommend doing this on both `push` and `pull_request` for maximum test coverage: ```yaml name: "TestingBot Test" on: [push, pull_request] jobs: test: runs-on: ubuntu-latest name: Sample Test ... ``` You can specify a `runs-on` parameter, to indicate the OS of the container that should run your test. The most important part is the `steps` section, which we'll cover in the example below. ## Example workflow with TestingBot Let's set up a workflow that will do the following actions: - Checkout the current GitHub repository code - Install dependencies for the project - Run the tests in the repository ```yaml name: "PR Checks" on: [push, pull_request] jobs: test: runs-on: ubuntu-latest name: Action Test steps: - uses: actions/checkout@v4 - uses: testingbot/testingbot-tunnel-action@v1 with: key: ${{ secrets.TB_KEY }} secret: ${{ secrets.TB_SECRET }} tunnelIdentifier: github-action-tunnel - name: "Install Dependencies" run: npm install - name: "Run Test" run: npm run test env: TB_KEY: ${{ secrets.TB_KEY }} TB_SECRET: ${{ secrets.TB_SECRET }} TB_BUILD: github-${{ github.run_id }}-${{ github.run_attempt }} ``` Here we're including the `testingbot/testingbot-tunnel-action` TestingBot GitHub action with a `key`, `secret` and `tunnelIdentifier`. The `tunnelIdentifier` is used to indicate in your tests that a connection should be made to this specific tunnel. We also expose a `TB_BUILD` environment variable, built from the [GitHub run context](https://docs.github.com/en/actions/learn-github-actions/contexts) (`github.run_id` and `github.run_attempt`). By passing this as the `build` capability in your tests (see below), every browser session started in this workflow run is grouped into a single TestingBot [**build**](https://testingbot.com/support/other/status-badges#builds). This lets you report one combined pass/fail status back to the commit or Pull Request. ### Example Test For example, let's say your `npm run test` starts a local `http-server` and then runs a `Selenium WebDriver` test: ```bash http-server ./test -p 8080 > /dev/null 2>&1 & ``` [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) ```ruby #!/usr/bin/env ruby require 'rubygems' require 'selenium-webdriver' options = Selenium::WebDriver::Chrome::Options.new options.add_option('platformName', 'WIN11') options.add_option('browserVersion', 'latest') options.add_option('tb:options', { 'key' => ENV['TB_KEY'], 'secret' => ENV['TB_SECRET'], 'name' => 'GitHub Action Test', 'build' => ENV['TB_BUILD'], 'tunnel-identifier' => 'github-action-tunnel' }) driver = Selenium::WebDriver.for( :remote, url: "https://hub.testingbot.com/wd/hub", options: options) driver.navigate.to "http://localhost:8080" puts driver.title driver.quit ``` ```java import org.openqa.selenium.WebDriver; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URL; import java.util.HashMap; import java.util.Map; public class AccessibilityTest { public static final String KEY = System.getenv("TB_KEY"); public static final String SECRET = System.getenv("TB_SECRET"); public static final String URL = "https://hub.testingbot.com/wd/hub"; public static void main(String[] args) throws Exception { ChromeOptions options = new ChromeOptions(); options.setCapability("browserVersion", "latest"); options.setCapability("platformName", "WIN11"); Map tbOptions = new HashMap<>(); tbOptions.put("key", KEY); tbOptions.put("secret", SECRET); tbOptions.put("name", "GitHub Action Test"); tbOptions.put("build", System.getenv("TB_BUILD")); tbOptions.put("tunnel-identifier", "github-action-tunnel"); options.setCapability("tb:options", tbOptions); WebDriver driver = new RemoteWebDriver(new URL(URL), options); driver.get("http://localhost:8080"); System.out.println(driver.getTitle()); driver.quit(); } } ``` ```php setCapability('platformName', 'WIN11'); $capabilities->setCapability('browserVersion', 'latest'); $capabilities->setCapability('tb:options', [ 'key' => $tbKey, 'secret' => $tbSecret, 'name' => 'GitHub Action Test', 'build' => getenv('TB_BUILD'), 'tunnel-identifier' => 'github-action-tunnel' ]); $web_driver = RemoteWebDriver::create( "https://hub.testingbot.com/wd/hub", $capabilities, 120000 ); $web_driver->get("http://localhost:8080"); echo $web_driver->getTitle(); $web_driver->quit(); ?> ``` ```python import os from selenium import webdriver from selenium.webdriver.chrome.options import Options options = Options() options.set_capability('platformName', 'WIN11') options.set_capability('browserVersion', 'latest') options.set_capability('tb:options', { 'key': os.environ['TB_KEY'], 'secret': os.environ['TB_SECRET'], 'name': 'GitHub Action Test', 'build': os.environ.get('TB_BUILD'), 'tunnel-identifier': 'github-action-tunnel' }) driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options ) driver.get("http://localhost:8080") print(driver.title) driver.quit() ``` ```javascript const webdriver = require('selenium-webdriver'); const chrome = require('selenium-webdriver/chrome'); const testingbotKey = process.env.TB_KEY; const testingbotSecret = process.env.TB_SECRET; async function runGitHubActionTest () { let options = new chrome.Options(); options.set('platformName', 'WIN11'); options.set('browserVersion', 'latest'); options.set('tb:options', { 'key': testingbotKey, 'secret': testingbotSecret, 'name': 'GitHub Action Test', 'build': process.env.TB_BUILD, 'tunnel-identifier': 'github-action-tunnel' }); let driver = new webdriver.Builder() .usingServer('https://hub.testingbot.com/wd/hub') .withCapabilities(options) .build(); await driver.get("http://localhost:8080"); const title = await driver.getTitle(); console.log(title); await driver.quit(); } runGitHubActionTest(); ``` ```csharp using System; using System.Collections.Generic; using OpenQA.Selenium; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Remote; namespace SeleniumTest { class Program { static void Main(string[] args) { var options = new ChromeOptions(); options.PlatformName = "WIN11"; options.BrowserVersion = "latest"; options.AddAdditionalOption("tb:options", new Dictionary { { "key", Environment.GetEnvironmentVariable("TB_KEY") }, { "secret", Environment.GetEnvironmentVariable("TB_SECRET") }, { "name", "GitHub Action Test" }, { "build", Environment.GetEnvironmentVariable("TB_BUILD") }, { "tunnel-identifier", "github-action-tunnel" } }); IWebDriver driver = new RemoteWebDriver( new Uri("https://hub.testingbot.com/wd/hub"), options ); driver.Navigate().GoToUrl("http://localhost:8080"); Console.WriteLine(driver.Title); driver.Quit(); } } } ``` Your test will connect to the local http-server via the TestingBot GitHub Action and print the title of the page. ## PR Status Checks with the GitHub App The TestingBot GitHub App posts a pass or fail `TestingBot / tests` status check on each pull request, with a browser and device result matrix and an optional merge gate. It works with the workflow above: register the run's build against the pull request and TestingBot posts the check. Set up GitHub PR Checks See the dedicated [GitHub PR Checks](https://testingbot.com/support/integrations/ci-cd/github-pr-checks) guide for the install steps, the workflow snippet, and how to make the check a required status check so broken builds cannot be merged. ## Build Status on a Pull Request Because every session in the workflow run shares the same `build` capability (the `TB_BUILD` value from the [example above](https://testingbot.com#example)), TestingBot groups them into a single **build**. When your test runner reports the pass/fail state of each test back to TestingBot, we automatically calculate the overall status of that build: - If every test in the build passed, the build is marked as **Passing**. - If any test failed, the build is marked as **Failed**. - If we don't receive a state for every test, the build status is **Unknown**. See [Marking a build as passed/failed](https://testingbot.com/support/other/status-badges#update) for how to send the success state of each test using the [TestingBot REST API](https://testingbot.com/support/api). ### Show a status badge You can embed a TestingBot [status badge](https://testingbot.com/support/other/status-badges) in your repository's `README` to show the result of your most recent build: Badge | Status || | **Passing** | | | **Failure** | **Markdown:** ```markdown [![TestingBot Test Status](https://testingbot.com/buildstatus/YOUR_TESTINGBOT_KEY?auth={authentication})](https://testingbot.com/members/builds) ``` ### Query a build's status from the API You can also look up the aggregate status of a specific build (for example, to add your own custom check step to a workflow). The [REST API](https://testingbot.com/support/api) returns whether the build passed, together with the number of total, passed and failed tests: ```bash curl -u "$TB_KEY:$TB_SECRET" \ https://api.testingbot.com/v1/builds/github-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT} ``` A failing build (one or more failed tests) lets you fail the workflow step, which surfaces as a failed check on the commit and Pull Request. ## Require Tests to Pass before Merging A popular workflow is to **block a Pull Request from being merged until all tests pass**. Make the check mandatory using [branch protection rules](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches): 1. In your GitHub repository, go to **Settings** → **Branches** (or **Rules** → **Rulesets** ). 2. Add a branch protection rule for your default branch (for example `main`). 3. Enable **Require status checks to pass before merging**. 4. Search for and select `TestingBot / tests` (the [GitHub App](https://testingbot.com#pr-checks) check) as a required check. If you are not using the app, select your TestingBot workflow job instead (for example `Action Test`). Once enabled, the **Merge** button stays disabled until the TestingBot build passes, so no Pull Request is merged with failing tests. An incomplete or never-finished build reports a failing check rather than a passing one, so a broken run cannot slip through. Check not showing up? A check only becomes selectable in the required-checks list after it has run at least once on a recent commit. Open or update a Pull Request first so the check appears. ## Inputs The TestingBot GitHub Action accepts the following inputs: | Input | Description | | --- | --- | | `key` **Required** | Your TestingBot API Key | | `secret` **Required** | Your TestingBot API Secret | | `auth` | Performs Basic Authentication for specific hosts, only works with HTTP. | | `debug` | Enables debug messages. Will output request/response headers. | | `dns` | Use a custom DNS server. For example: 8.8.8.8 | | `doctor` | Perform sanity/health checks to detect possible misconfiguration or problems. | | `fastFailRegexps` | Specify domains you don't want to proxy, comma separated. | | `pac` | Proxy autoconfiguration. Should be a http(s) URL | | `sePort` | The local port your Selenium test should connect to. Default port is 4445 | | `localProxy` | The port to launch the local proxy on (default 8087). | | `proxy` | Specify an upstream proxy: `PROXYHOST:PROXYPORT` | | `proxyCredentials` | Username and password required to access the proxy configured with `proxy`. | | `noCache` | Bypass TestingBot Caching Proxy running on the tunnel VM. | | `noProxy` | Do not start a local proxy (requires user provided proxy server on port 8087). | | `tunnelIdentifier` | Add an identifier to this tunnel connection. In case of multiple tunnels, specify this identifier in your desired capabilities to use this specific tunnel. | | `uploadLogFile` | Should this action [upload the log file](https://testingbot.com#artifacts) generated by the TestingBot Tunnel as an artifact? Default is true. | | `retryTimeout` | How long, in seconds, should the Action wait to retry, if the tunnel fails to start. | ## Artifacts The TestingBot GitHub Action will, by default, upload the logfile generated by the TestingBot tunnel. This allows you to view what happened with the TestingBot Tunnel for each of your Actions. The logfile is saved as an artifact and can be downloaded. ![GitHub Artifact](https://testingbot.com/assets/support/github/artifact-9ec2869f7e562823c0fac8784cbde1a2fc0820830e31410393405b00bed32d49.webp) If you don't want this, you can disable it by setting the input `uploadLogFile: "false"` in your workflow file. ## More Information More information about this is available on the [GitHub Documentation pages](https://docs.github.com/en/actions) and our [TestingBot GitHub Action repository](https://github.com/testingbot/testingbot-tunnel-action). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/ci-cd/github-pr-checks # GitHub PR Status Checks The TestingBot GitHub App posts a single `TestingBot / tests` status check on each pull request. The check shows a browser and device result matrix with a deep link to every failing session's video and logs, a short AI summary of why the failing tests failed, and it can be made a [required status check](https://testingbot.com#merge-gate) so a pull request cannot be merged while your TestingBot tests are failing. This works alongside the [TestingBot GitHub Action](https://testingbot.com/support/integrations/ci-cd/github-actions) (or any CI that runs your TestingBot tests): you group a run's sessions under one build, then register that build against the pull request so its result becomes the check. [Maestro runs](https://testingbot.com#maestro) need no registration step at all: the TestingBot CLI and the Maestro GitHub Action pass the commit along, and the check follows automatically. ## Install the GitHub App 1. In the members area, open [Integrations](https://testingbot.com/members/integrations) and choose **GitHub PR Checks**. 2. Click **Install the TestingBot GitHub App**. You will be redirected to GitHub. 3. Pick the account or organization and the repositories to enable, then install. You are redirected back and the integration shows as **Connected**. You can change the repository selection or uninstall the app from your GitHub settings at any time. Only account owners and admins can connect or disconnect the integration. ## Register the build from your workflow Give every session in a run the same `build` capability (the example workflow uses `TB_BUILD`), then add two steps around your test job. The first registers the build against the pull request's head commit and posts an in-progress check; the second marks the build finished so the check resolves to pass or fail. Pass `github.event.pull_request.head.sha` as the commit (not `github.sha`, which is the merge commit) so the check lands on the commit GitHub evaluates for merging. ```yaml jobs: test: runs-on: ubuntu-latest env: TB_KEY: ${{ secrets.TESTINGBOT_KEY }} TB_SECRET: ${{ secrets.TESTINGBOT_SECRET }} TB_BUILD: github-${{ github.run_id }}-${{ github.run_attempt }} steps: - uses: actions/checkout@v4 - name: Start TestingBot PR check if: github.event_name == 'pull_request' run: | curl -sf -u "$TB_KEY:$TB_SECRET" \ -X POST "https://api.testingbot.com/v1/builds/$TB_BUILD/ci" \ -d "repo=${{ github.repository }}" \ -d "commit_sha=${{ github.event.pull_request.head.sha }}" \ -d "pull_request_number=${{ github.event.pull_request.number }}" - name: Run your TestingBot tests run: | # your existing test command, using build: $TB_BUILD in the capabilities - name: Resolve TestingBot PR check if: ${{ always() && github.event_name == 'pull_request' }} run: | curl -sf -u "$TB_KEY:$TB_SECRET" \ -X PUT "https://api.testingbot.com/v1/builds/$TB_BUILD/ci" ``` The check appears as in-progress the moment you register the build, then resolves once you signal completion. If the workflow stops before the resolve step runs, TestingBot finalizes the check automatically after a short grace period, so a pull request is never left waiting on a check that will never arrive. ![TestingBot / tests status check failing on a GitHub pull request, showing 3 of 5 tests failed](https://testingbot.com/assets/support/github-pr-checks/pr-check-1e5054cfb6a7fc84eed29a1b7480ca8d1829d71086bf641d8fceb5065036298f.png) Use the head commit SHA For `pull_request` runs, always pass `github.event.pull_request.head.sha`. Passing `github.sha` posts the check to the merge commit, which is not part of the pull request, so the check appears to be missing and the PR cannot be merged. ## Maestro runs Maestro runs started with the TestingBot CLI or the Maestro GitHub Action get the same `TestingBot / tests` check without the register and resolve steps above: the run carries the repository and head commit itself, and TestingBot finalizes the check once every flow on every device has finished. The check lists each flow per device and the sticky comment carries the same table. Setup, the merge-gate rules and troubleshooting are on [GitHub PR Checks for Maestro](https://testingbot.com/support/app-automate/maestro/ci-cd/github-pr-checks). ![TestingBot / tests check on a GitHub pull request for a Maestro run, with the device, flow and result table](https://testingbot.com/assets/support/github-pr-checks/maestro-check-6185df5c10dcff0db4042e811f527a36ba354a07a1fb30f4a9ebb6b2e7c5cd65.png) ## Require tests to pass before merging To block a pull request from merging until your TestingBot tests pass, make the check mandatory with [branch protection rules](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches): 1. In your GitHub repository, go to **Settings** → **Branches** (or **Rules** → **Rulesets** ). 2. Add a rule for your default branch (for example `main`). 3. Enable **Require status checks to pass before merging**. 4. Search for and select `TestingBot / tests` as a required check. Once enabled, the **Merge** button stays disabled until the TestingBot build passes on the pull request's latest commit. ## How the pass/fail result is decided The check reflects the aggregate result of every session in the build: - **Passing** when every test in the build passed. - **Failing** when any test failed. - **Failing** when the build is incomplete or reported no definitive result. This is deliberate: a stuck or undetermined build blocks the merge instead of silently allowing it, because GitHub treats a neutral check as passing. In other words, the check is green only when the build is genuinely green. A workflow that dies before signalling completion is finalized as failing by the grace-period watchdog rather than left pending forever. ## Pull request comment and AI summary Alongside the check, TestingBot maintains a single, self-updating comment on the pull request with the full browser and device result matrix. Failing environments are listed first, each with a deep link to that session's video, logs and step timeline. Pushing new commits updates the same comment rather than adding new ones. ![TestingBot pull request comment showing a browser and device result matrix with pass and fail per environment](https://testingbot.com/assets/support/github-pr-checks/pr-comment-d595e8553666f57df9766dbce29338e63883ca2cc63f5f982e8a6d7505c6ca67.png) When tests fail, TestingBot adds a short **"Why these failed"** section that uses AI to explain the most likely cause of the top failing tests, right in the check and the comment, so you can triage without leaving the pull request. The check also carries a **Re-run analysis** button to re-evaluate the current build result on demand. ## Troubleshooting - **The check does not appear on the pull request.** Make sure you passed the pull request's head SHA (`github.event.pull_request.head.sha`), not the merge commit, and that the app is installed on the repository. - **The check is not selectable as a required check.** GitHub only lists a check as a required-check option after it has run at least once on the protected branch within the last 7 days. Open one pull request so the check runs, then add it to your branch protection rule. - **The check is stuck as in-progress.** Confirm your workflow calls the `PUT` step when the run finishes. If the workflow was cancelled, the watchdog finalizes the check as failing after a short grace period. - **Nothing posts at all.** Verify the integration shows as **Connected** under [GitHub PR Checks](https://testingbot.com/members/integrations/github-app) and that the repository owner matches the account the app was installed on. - **A Maestro run shows no check.** The CLI logs `GitHub check registered` when the link was made. Without that line, check that `--repo-owner`, `--repo-name` and a full 40-character `--commit-sha` were passed (the Maestro GitHub Action does this for you) and that the app is installed for the repository owner. --- URL: https://testingbot.com/support/integrations/ci-cd/gitlab # GitLab CI Automated Testing GitLab CI/CD is a part of GitLab and provides a Continuous Integration system to run automated tests. Below are the steps to run your Selenium tests via GitLab CI, once the tests are up and running you can even display a TestingBot Status badge showcasing your Selenium tests status. ## 1. Get GitLab CI up and running Download [GitLab CI](https://about.gitlab.com/product/continuous-integration/) and set up GitLab CI to your requirements. ## 2. Create a simple Selenium test Add a simple Selenium test to your GitLab project, modify the .gitlab-ci.yml file in your repository to indicate you want to run a Selenium test. Example `.gitlab-ci.yml` file: ```yaml image: node:latest stages: - Test TestingBot Selenium Test: stage: Test TestingBot before_script: - npm install - npm install selenium-webdriver script: - node sample_test.js ``` Add your `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` environment variables to [GitLab CI/CD variables](https://docs.gitlab.com/ee/ci/variables/). ```javascript const webdriver = require('selenium-webdriver'); const chrome = require('selenium-webdriver/chrome'); async function runSampleTest () { let driver; try { let options = new chrome.Options(); options.set('platformName', 'WIN11'); options.set('browserVersion', 'latest'); options.set('tb:options', { 'key': process.env.TESTINGBOT_KEY, 'secret': process.env.TESTINGBOT_SECRET, 'name': (process.env.CI_JOB_ID ? ("GitLab Build " + process.env.CI_JOB_ID) : "Simple Test") }); driver = new webdriver.Builder() .usingServer("https://hub.testingbot.com/wd/hub") .withCapabilities(options) .build(); await driver.get('https://www.google.com'); var title = await driver.getTitle(); console.log("title is: " + title); } catch (e) { console.log(e); } finally { if (driver) { await driver.quit(); } } } runSampleTest(); ``` Now you're ready to run this first sample test with GitLab CI. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/ci-cd/gradle # Gradle Plugin We maintain a [Gradle](https://gradle.org/) plugin that uploads your Android app to [TestingBot Storage](https://testingbot.com/support/app-automate/help/upload) and runs [Espresso](https://testingbot.com/support/app-automate/espresso) tests on TestingBot's real device cloud, straight from your build, with no `curl` scripts required. - `testingbotUpload` uploads an APK to TestingBot Storage and returns a reusable `tb://` identifier. - `testingbotEspresso` uploads your app and instrumented test APK, runs Espresso across one or more devices, waits for the results, downloads a JUnit XML report, and fails the build on test failures. - `testingbotDevices` lists the Android devices available on your account. - Reads credentials from the DSL, Gradle properties or environment variables, and is configuration-cache compatible with no extra HTTP dependencies on your build classpath. The plugin is published on the [Gradle Plugin Portal](https://plugins.gradle.org/plugin/com.testingbot.gradle) and the source is available on [GitHub](https://github.com/testingbot/testingbot-gradle-plugin). ## Requirements - Gradle 8.0 or newer, running on JDK 17 or newer. - A TestingBot account. Grab your **key** and **secret** from the [member area](https://testingbot.com/members/user/edit). ## 1. Installation Apply the plugin in your Android app module's build file. ```kotlin // build.gradle.kts plugins { id("com.testingbot.gradle") version "0.1.0" } ``` Using the Groovy DSL: ```groovy // build.gradle plugins { id 'com.testingbot.gradle' version '0.1.0' } ``` ## 2. Credentials Credentials are resolved in this order, first match wins: - The DSL: `testingbot { key.set("…"); secret.set("…") }` - Gradle properties: `-Ptestingbot.key=… -Ptestingbot.secret=…` (or in `gradle.properties`) - Environment variables: `TESTINGBOT_KEY` / `TESTINGBOT_SECRET` In CI, prefer environment variables backed by secrets. Never commit credentials to your repository. ## 3. Usage Configure the `testingbot { }` block with your APKs and the device capabilities you want to run on. ```kotlin testingbot { // Credentials (optional here if provided via env vars / gradle properties) key.set(providers.environmentVariable("TESTINGBOT_KEY")) secret.set(providers.environmentVariable("TESTINGBOT_SECRET")) // The app + instrumented test APKs appApk.set(layout.projectDirectory.file("app/build/outputs/apk/debug/app-debug.apk")) testApk.set( layout.projectDirectory.file("app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk") ) // One run is started per capability. Keys are passed through to TestingBot verbatim. capabilities.set( listOf( mapOf("deviceName" to "Pixel.*", "version" to "14", "platformName" to "Android", "realDevice" to true), mapOf("deviceName" to "Galaxy S.*", "version" to "13", "platformName" to "Android") ) ) // Optional behavior waitForResults.set(true) // poll until the run completes (default true) failBuildOnTestFailure.set(true) // default true reportXml.set(layout.buildDirectory.file("testingbot/espresso-junit.xml")) } ``` `deviceName` accepts a regular expression, so `"Pixel.*"` matches any Pixel and `"*"` matches any device. Commonly used keys are `deviceName`, `version`, `platformName` (`"Android"`), `realDevice`, `phoneOnly` and `tabletOnly`. Run `./gradlew testingbotDevices` to see what is available on your plan. ## 4. Tasks ```bash # Upload only ./gradlew testingbotUpload # Build the APKs first, then run Espresso on TestingBot ./gradlew assembleDebug assembleDebugAndroidTest testingbotEspresso # Discover device names/versions for your capabilities ./gradlew testingbotDevices ``` ## 5. Android Gradle Plugin auto-wiring If the `com.android.application` plugin is applied, naming a variant lets the plugin fill in the conventional APK paths and depend on the matching `assemble` tasks automatically. ```kotlin testingbot { autoWireFromVariant.set("debug") // defaults appApk/testApk + dependsOn assembleDebug(AndroidTest) capabilities.set(listOf(mapOf("deviceName" to "Pixel.*", "version" to "14", "platformName" to "Android"))) } ``` Now `./gradlew testingbotEspresso` builds the APKs and runs the tests in one go. Explicitly set `appApk` / `testApk` paths always take precedence. ## Full DSL reference | Property | Type | Default | Description | | --- | --- | --- | --- | | `key` | `String` | `TESTINGBOT_KEY` / `testingbot.key` | API key | | `secret` | `String` | `TESTINGBOT_SECRET` / `testingbot.secret` | API secret | | `apiBaseUrl` | `String` | `https://api.testingbot.com/v1` | API base URL | | `appApk` | `RegularFile` | — | App APK to upload / test | | `testApk` | `RegularFile` | — | Instrumented Espresso test APK | | `capabilities` | `List>` | `[]` | One run per entry; passed through verbatim | | `espressoOptions` | `Map` | `{}` | Optional run options passed through verbatim | | `waitForResults` | `Boolean` | `true` | Poll until the run completes | | `pollIntervalSeconds` | `Long` | `15` | Seconds between status polls | | `timeoutMinutes` | `Long` | `30` | Max minutes to wait for results | | `failBuildOnTestFailure` | `Boolean` | `true` | Fail the build when a run reports failures | | `reportXml` | `RegularFile` | `build/testingbot/espresso-junit.xml` | Where to write the JUnit report | | `autoWireFromVariant` | `String` | — | Optional AGP variant to auto-wire | ## CI example (GitHub Actions) ```yaml - uses: actions/setup-java@v4 with: { distribution: temurin, java-version: 21 } - run: ./gradlew assembleDebug assembleDebugAndroidTest testingbotEspresso env: TESTINGBOT_KEY: ${{ secrets.TESTINGBOT_KEY }} TESTINGBOT_SECRET: ${{ secrets.TESTINGBOT_SECRET }} ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/ci-cd/jenkins # TestingBot Jenkins Plugin This guide will help you integrate the [TestingBot Jenkins Plugin](https://plugins.jenkins.io/testingbot) in your Jenkins jobs and pipelines. The plugin integrates the TestingBot browser and device cloud directly into Jenkins: - Stores your TestingBot key and secret through the Jenkins Credentials plugin, with a **Test Connection** button to verify them. - Starts a [TestingBot Tunnel](https://testingbot.com/support/tunnel) around your build, so your tests can reach servers behind your firewall or on the build agent. - Embeds the video, screenshots and logs of every test on the build page, so you never have to leave Jenkins to review a run. - Reports each test's pass/fail result to TestingBot, and can publish the outcome as a GitHub check on the commit and pull request. - Supports both freestyle jobs and Pipeline (Jenkinsfile), as well as [Configuration as Code](https://testingbot.com#jcasc). ![TestingBot test reports embedded in a Jenkins build](https://testingbot.com/assets/support/jenkins/postbuild-8ab841d1020664db1f778c470b81121b020c4782dda745d51fb8522f85798a5e.png) ## Prerequisites - A [TestingBot account](https://testingbot.com/members) with an API key and secret. - Jenkins **2.541.3** or newer, running on **Java 17** or newer. - The [JUnit plugin](https://plugins.jenkins.io/junit/) to embed per-test reports (usually already installed). ## Install the Plugin In Jenkins, go to **Manage Jenkins** \> **Plugins** \> **Available plugins** , search for **TestingBot** , select it and install. You can install without restarting Jenkins. The plugin is also listed in the [Jenkins plugins directory](https://plugins.jenkins.io/testingbot). ## Add your Credentials Go to **Manage Jenkins** \> **Credentials** , add a credential of kind **TestingBot** , and enter your key and secret. You can obtain both values from the [TestingBot member area](https://testingbot.com/members/user/edit). Click **Test Connection** to verify the key and secret against TestingBot before saving — it confirms which account they belong to, so you catch a typo before your first build runs. ![Adding TestingBot credentials in Jenkins with a Test Connection button](https://testingbot.com/assets/support/jenkins/credentials-7ee0ec513872f4120d369afed89b5114677ffd819df51a8c09790da49b5595f8.png) ## Set up the Build Environment In a freestyle job's configuration, open the **Build Environment** section and enable **TestingBot**. Pick the credentials you added above; the plugin then injects your key and secret into the build as environment variables (see [Environment Variables](https://testingbot.com#env)). Enable **Use TestingBot Tunnel** to start a [TestingBot Tunnel](https://testingbot.com/support/tunnel) before the build and stop it afterwards, so tests running on the agent can reach internal, staging or `localhost` environments through the TestingBot grid. ![TestingBot Build Environment option in a Jenkins job](https://testingbot.com/assets/support/jenkins/buildenv-78f244e57a14a8cc41274d3ee405ec3080ad6df3ea8f47e0f643bf386c645b85.png) If your tests target a staging or internal environment that sits behind a firewall, either enable the Tunnel above (recommended), or allowlist TestingBot's grid IP ranges on that environment so our browsers can reach it. The current ranges are listed on the [IP ranges](https://testingbot.com/support/other/configuration#ip-range) page. The Tunnel runs on the **agent** that executes the build, so if you pin TestingBot jobs to specific agents (via a node label and **Restrict where this project can be run** ), those agents need **Java 17 or newer** on their `PATH`. Each `testingbotTunnel { }` block starts an isolated tunnel with its own `TESTINGBOT_TUNNEL_IDENTIFIER`, so parallel builds on the same agent don't interfere. ## Example Test Point your test at the TestingBot grid and read the key and secret from the environment variables the plugin injects. Set the `build` capability to `$TESTINGBOT_BUILD` so all sessions from one Jenkins build are grouped together (see the [Build Report](https://testingbot.com#build-report) below). [Java](https://testingbot.com#) [Python](https://testingbot.com#) [Ruby](https://testingbot.com#) [NodeJS](https://testingbot.com#) ```java import org.openqa.selenium.WebDriver; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import java.net.URL; import java.util.HashMap; import java.util.Map; public class JenkinsTest { public static void main(String[] args) throws Exception { ChromeOptions options = new ChromeOptions(); options.setCapability("browserVersion", "latest"); options.setCapability("platformName", "WIN11"); Map tbOptions = new HashMap<>(); tbOptions.put("key", System.getenv("TB_KEY")); tbOptions.put("secret", System.getenv("TB_SECRET")); tbOptions.put("name", "Jenkins Test"); tbOptions.put("build", System.getenv("TESTINGBOT_BUILD")); tbOptions.put("tunnel-identifier", System.getenv("TESTINGBOT_TUNNEL_IDENTIFIER")); options.setCapability("tb:options", tbOptions); RemoteWebDriver driver = new RemoteWebDriver( new URL("https://hub.testingbot.com/wd/hub"), options); // Print the session id so the plugin can embed this test's report. System.out.println("TestingBotSessionID=" + driver.getSessionId().toString()); driver.get("http://localhost:8080"); System.out.println(driver.getTitle()); driver.quit(); } } ``` ```python import os from selenium import webdriver from selenium.webdriver.chrome.options import Options options = Options() options.set_capability('browserVersion', 'latest') options.set_capability('platformName', 'WIN11') options.set_capability('tb:options', { 'key': os.environ['TB_KEY'], 'secret': os.environ['TB_SECRET'], 'name': 'Jenkins Test', 'build': os.environ.get('TESTINGBOT_BUILD'), 'tunnel-identifier': os.environ.get('TESTINGBOT_TUNNEL_IDENTIFIER') }) driver = webdriver.Remote( command_executor='https://hub.testingbot.com/wd/hub', options=options) # Print the session id so the plugin can embed this test's report. print("TestingBotSessionID=" + driver.session_id) driver.get("http://localhost:8080") print(driver.title) driver.quit() ``` ```ruby #!/usr/bin/env ruby require 'rubygems' require 'selenium-webdriver' options = Selenium::WebDriver::Chrome::Options.new options.add_option('browserVersion', 'latest') options.add_option('platformName', 'WIN11') options.add_option('tb:options', { 'key' => ENV['TB_KEY'], 'secret' => ENV['TB_SECRET'], 'name' => 'Jenkins Test', 'build' => ENV['TESTINGBOT_BUILD'], 'tunnel-identifier' => ENV['TESTINGBOT_TUNNEL_IDENTIFIER'] }) driver = Selenium::WebDriver.for( :remote, url: "https://hub.testingbot.com/wd/hub", options: options) # Print the session id so the plugin can embed this test's report. puts "TestingBotSessionID=" + driver.session_id driver.navigate.to "http://localhost:8080" puts driver.title driver.quit ``` ```javascript const webdriver = require('selenium-webdriver'); const chrome = require('selenium-webdriver/chrome'); async function runJenkinsTest () { let options = new chrome.Options(); options.set('browserVersion', 'latest'); options.set('platformName', 'WIN11'); options.set('tb:options', { 'key': process.env.TB_KEY, 'secret': process.env.TB_SECRET, 'name': 'Jenkins Test', 'build': process.env.TESTINGBOT_BUILD, 'tunnel-identifier': process.env.TESTINGBOT_TUNNEL_IDENTIFIER }); let driver = new webdriver.Builder() .usingServer('https://hub.testingbot.com/wd/hub') .withCapabilities(options) .build(); // Print the session id so the plugin can embed this test's report. const session = await driver.getSession(); console.log("TestingBotSessionID=" + session.getId()); await driver.get("http://localhost:8080"); console.log(await driver.getTitle()); await driver.quit(); } runJenkinsTest(); ``` A complete, runnable project is available on our [Jenkins-Demo](https://github.com/testingbot/Jenkins-Demo) repository. ## Mobile / Native App Testing To test a native mobile app (Appium) that your Jenkins job builds, upload the app to [TestingBot Storage](https://testingbot.com/support/api/storage) and reference the returned `tb://` app URL as the Appium `app` capability. Android `.apk`, iOS `.ipa`, and a `.zip` of an iOS Simulator `.app` build are all supported. In a **freestyle** job, add the **Upload an app to TestingBot Storage** build step _before_ your test step and point it at the built app. It uploads the file and exports the app URL as an environment variable (default `TESTINGBOT_APP_URL`) for the following steps to use. In a **pipeline** , the `testingbotUpload` step returns the app URL: ```groovy testingbot('251ca561abdfewf285') { def appUrl = testingbotUpload(file: 'build/app.apk') // pass appUrl as the Appium "app" capability in your tests sh "APP_URL=${appUrl} ./run-appium-tests.sh" } ``` The upload runs on the agent that holds the artifact, so only the credentials cross the (encrypted) remoting link — the app bytes are sent straight from the agent to TestingBot. From there, embedding reports, grouping sessions under `$TESTINGBOT_BUILD` and publishing [GitHub checks](https://testingbot.com#checks) work exactly as they do for browser tests. ## Embed TestingBot Reports To see the video, screenshots and logs of each test on the build page, the plugin reads the `TestingBotSessionID=` lines your tests print (as shown in the example above) and matches them against your JUnit results. Add the **Publish JUnit test result report** post-build action and point it at your report files (for example `test-reports/*.xml`). Then, under **Additional test report features** , add **Embed TestingBot reports**. Besides embedding the media, this also marks each session as **passed** or **failed** on TestingBot, based on the outcome of the matching JUnit test — so the status you see in Jenkins matches the status on TestingBot. In a pipeline, the `testingbotPublisher()` step does the same. The plugin parses both `stdout` and `stderr` for the `TestingBotSessionID=` token. When using JUnit reporting, the token needs to appear in a `` tag of the JUnit XML. On the build page, each JUnit test that used TestingBot shows a preview thumbnail of its session that expands in place to the full video, screenshots and logs — the media loads lazily, so collapsed sessions stay lightweight. ![Embed TestingBot reports post-build action in Jenkins](https://testingbot.com/assets/support/jenkins/postbuild-8ab841d1020664db1f778c470b81121b020c4782dda745d51fb8522f85798a5e.png) ## Embedded Build Report Whenever the plugin injects credentials — through the freestyle Build Environment option or the `testingbot { }` / `testingbotTunnel { }` pipeline steps — it exposes a `TESTINGBOT_BUILD` environment variable and adds a **TestingBot Build** page to the build. Set your test's `build` capability to `$TESTINGBOT_BUILD` so that all sessions from a single Jenkins build are grouped under one TestingBot build. The **TestingBot Build** link on the build page then embeds that build's report — every session, with status and video — directly inside Jenkins. ## Publish Results as a GitHub Check The plugin can publish the TestingBot outcome of a build as a GitHub check (✅/❌ on the commit and pull request) through the [Checks API](https://plugins.jenkins.io/checks-api/) plugin. Add the **Publish TestingBot results as a GitHub check** post-build action in a freestyle job, or call the `testingbotChecks` step in a pipeline: ```groovy testingbotChecks(name: 'TestingBot', message: 'End-to-end tests on TestingBot') ``` - `name` — the check name (its _context_ on the commit/PR). Defaults to `TestingBot`. - `message` — an optional summary; when omitted, a summary is generated from the TestingBot sessions in the build. The check links to the embedded TestingBot build report. The check conclusion reflects the TestingBot sessions found in the build (all passed → success), falling back to the overall build result when no sessions are present. Delivery to GitHub is handled by the [GitHub Checks](https://plugins.jenkins.io/github-checks/) plugin together with a GitHub App; if that is not installed, the step is a safe no-op. ![TestingBot results published as a GitHub check](https://testingbot.com/assets/support/jenkins/testingbot-checks-cdd334d8a5fca756af488f24361b297e60014295f4a9fff34f0e89e21f941cf8.png) ## Pipeline The plugin provides Pipeline (Jenkinsfile) support through these steps: - `testingbot(String credentialId)` — injects your TestingBot key and secret as environment variables for the wrapped block. - `testingbotTunnel(credentialsId: '', options: '-d -a')` — starts a TestingBot Tunnel around the wrapped block and stops it afterwards. `options` are passed verbatim to the [tunnel](https://testingbot.com/support/tunnel/commandline). - `testingbotUpload(file: 'build/app.apk')` — uploads a native app to [TestingBot Storage](https://testingbot.com#mobile) and returns its `tb://` app URL. - `testingbotPublisher()` — reads the JUnit report files and embeds the TestingBot test results. - `testingbotChecks(name: '', message: '')` — publishes the outcome as a [GitHub check](https://testingbot.com#checks). The `credentialId` is the Id shown on the Jenkins Credentials page for the TestingBot key and secret you added earlier. You can build any of these steps interactively with the Jenkins **Pipeline Syntax** (Snippet Generator). ![Jenkins Pipeline Snippet Generator for TestingBot steps](https://testingbot.com/assets/support/jenkins/snippet-generator-35353f524096b4c93725f8761838f4b9cb59c37135daa0154ad3ba2e422b1045.png) ### Declarative Pipeline ```groovy pipeline { agent any tools { // Install the Maven version configured as "M3" and add it to the path. maven "M3" ant "ant" } stages { stage('Build') { steps { // Get some code from a GitHub repository git 'https://github.com/testingbot/Jenkins-Demo.git' testingbot('251ca561abdfewf285') { testingbotTunnel(credentialsId: '251ca561abdfewf285', options: '-d') { sh "ant test" } } } post { success { junit 'test-reports/*.xml' } always { testingbotPublisher() } } } } } ``` ### Scripted Pipeline ```groovy node { git 'https://github.com/testingbot/Jenkins-Demo.git' // Inject credentials and start an isolated tunnel around the tests testingbot('251ca561abdfewf285') { testingbotTunnel(credentialsId: '251ca561abdfewf285', options: "--tunnel-identifier ci-${env.BUILD_NUMBER}") { sh 'ant test' } } junit 'test-reports/*.xml' // Embed TestingBot screenshots/video into the test report testingbotPublisher() } ``` Each `testingbotTunnel { }` block starts an isolated tunnel with its own identifier, so parallel pipeline branches never interfere with one another. ## Environment Variables Inside **both** `testingbot { }` and `testingbotTunnel { }` blocks (and in a freestyle job with the TestingBot Build Environment enabled): | Variable | Description | | --- | --- | | `TESTINGBOT_KEY` / `TB_KEY` | Your TestingBot API key. | | `TESTINGBOT_SECRET` / `TB_SECRET` | Your TestingBot API secret (masked in the build log). | | `TESTINGBOT_BUILD` | A per-build identifier. Pass it as your test's `build` capability so all sessions from this Jenkins build are grouped together and shown on the embedded [TestingBot Build](https://testingbot.com#build-report) page. | Inside a `testingbotTunnel { }` block only (these describe the tunnel started for that block): | Variable | Description | | --- | --- | | `HUB_HOST` / `HUB_PORT` | Host and port to point your Selenium or Appium client at when using the tunnel. | | `SELENIUM_HOST` / `SELENIUM_PORT` | Selenium-specific aliases of `HUB_HOST`/`HUB_PORT`, kept for backwards compatibility. | | `TESTINGBOT_TUNNEL_IDENTIFIER` | The identifier of the tunnel started for this block. Pass it in your desired capabilities so parallel builds each use their own tunnel. | ## Configuration as Code (JCasC) The TestingBot credentials can be configured with the [Configuration as Code](https://plugins.jenkins.io/configuration-as-code/) plugin using the `testingbot` symbol: ```yaml credentials: system: domainCredentials: - credentials: - testingbot: id: "testingbot" description: "TestingBot key/secret" key: "${TESTINGBOT_KEY}" secret: "${TESTINGBOT_SECRET}" ``` Reference the `id` (here `testingbot`) as the `credentialsId` in your job or pipeline. ## More Information The plugin is open source and listed in the [Jenkins plugins directory](https://plugins.jenkins.io/testingbot). More information, including framework examples and the full changelog, is available in the [TestingBot Jenkins Plugin repository](https://github.com/jenkinsci/testingbot-plugin). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/ci-cd/teamcity # TeamCity Automated Testing [TeamCity](https://www.jetbrains.com/teamcity/) is a Java-based build management/CI system created by Jetbrains. ## TeamCity Plugin We've created a plugin which offers integration with our TestingBot.com services. Below is a list of features this plugin offers: - Add a tab where we show the lists of tests for the current build. The overview shows a list of tests with their **passed/failed state** , together with the **browser they ran on**. - View **logs** (Selenium, Chrome, Firefox, Safari logs) and a **recorded video** together with **screenshots** of every test, straight from inside TeamCity. - Option to start/stop a [TestingBot Tunnel](https://testingbot.com/support/tunnel) automatically before and after every build run. - Predefined environment variables with your TestingBot `key` and `secret`. - Each build is **tagged** with the TestingBot session IDs it ran, so you can find a build back by session. - Your TestingBot secret is registered with TeamCity's password replacer, so it is **masked in the build log** even if a build step prints it. The plugin requires **TeamCity 2024.12** or newer, and Java 11 or newer on the build agent. ## 1. Install the Plugin Go to **Administration** and click **Plugins** , under **Server Administration**. ![TeamCity integration](https://testingbot.com/assets/support/teamcity/1-a868f01c08bb08e04d9b1399962e69ad1044522334bc99a31090f9f3e852aa18.png) ## 2. Upload Plugin Click **Upload plugin zip** and upload our TestingBot plugin. Or download the plugin from the [TeamCity plugin page](https://plugins.jetbrains.com/plugin/10283-testingbot/), or from the [releases page on GitHub](https://github.com/testingbot/testingbot-teamcity-plugin/releases). ![TeamCity integration](https://testingbot.com/assets/support/teamcity/2-d6f082e24dc8a03b75734bc38cdbe73dd49193f934f5041fb3298150362028aa.png) ## 3. Restart TeamCity Restart TeamCity to load the plugin. Once TeamCity is back up, the TestingBot plugin appears under **External plugins** on the same page. ## 4. Configuring a Project Now that the plugin is installed, we can use the TestingBot plugin in a TeamCity project. 1. Click on **Build Features** 2. Click **Add build feature** and select **TestingBot** from the list. ![TeamCity integration](https://testingbot.com/assets/support/teamcity/3-558f23bac8ac9771c1c35c7b28efa1bb590f3ca5c3efa44c3885533bed0f27fe.png) ## 5. Credentials Add your TestingBot **Key** and **Secret** , which you can obtain from the member dashboard. Both are required and TeamCity will not let you save the build feature while either is empty. Tick **Enable TestingBot Tunnel** if your tests need to reach a website that is not available on the public internet. The plugin starts a [TestingBot Tunnel](https://testingbot.com/support/tunnel) before the first build step, waits until it is ready, streams its output into the build log, and shuts it down when the build finishes. ![TeamCity integration](https://testingbot.com/assets/support/teamcity/4-7e43d73b3353e317b308e8cb9c3d3445de444aa989d3fe56a6e75f97912fe6b4.png) ## 6. Test Overview in TestingBot Tab You can now see an overview of your Selenium tests in the **TestingBot tab**. ![TeamCity integration](https://testingbot.com/assets/support/teamcity/7-3e978be89ee3b9458ad771c1f8edb95842903a34ef160c0d06d9166068f807bf.png) ## 7. Test Details Click a test to see a **video recording** , **screenshots** and **logs** for the test. ![TeamCity integration](https://testingbot.com/assets/support/teamcity/8-9b9cef361d30b29b05fe23985a96cf93ab46ef1450e022788ea4a62a22f605b2.png) ## 8. Configuring Tests The plugin will automatically set the following **Environment Variables** , which you can use in your tests: - `TB_KEY` : Your TestingBot key, available in our member area. - `TB_SECRET` : Your TestingBot secret, available in our member area. TeamCity masks this value in the build log. - `BUILD_ID` : An identifier for the current build, combining the build configuration ID and the build number. Set this as the `build` capability in your tests to group them together on TestingBot. **Important!** It is important to output `TestingBotSessionID={webdriver-session-id}` for every test to stdout. The plugin will parse the output looking for these ids. An example on how to do this with Java: ```java System.out.println("TestingBotSessionID=" + (((RemoteWebDriver) driver).getSessionId()).toString()); ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/integrations/ci-cd/travis-ci # Travis CI Automated Testing Travis CI is a continuous integration service which you can use to test your GitHub projects. It is straightforward to use the TestingBot Selenium grid together with Travis CI. Below are the steps to run your Selenium tests via Travis CI, once the tests are up and running you can even display a TestingBot Status badge showcasing your Selenium tests status. ## 1. Get Travis CI up and running Sign up at [Travis CI](https://www.travis-ci.com/) and connect your GitHub project with Travis CI. You can find information on how to do this at the Travis CI [help section](https://docs.travis-ci.com/). ## 2. Create a simple Selenium test Add a simple Selenium test to your GitHub project, modify the `.travis.yml` file in your repository to indicate you want to run a Selenium test. Example `.travis.yml` file: ```yaml language: node_js node_js: - 22 env: - [ {secure: "akuE0dld1Ke9mahjUUrQhUZRYWasdfewfwefewfZlbvOx\nqaPybirPGsDmImvcktaAkjLxpePd0V1+ak+4dws7dTrFfEsdvsdsdvsdvds2\nud1q5oGOzEqfiRGxY/fJHLWlaQ609Bsdfdsfds2VeY1Z/V7N9iQ="}, {secure: "JGfkAfr/SOlzV+NpgNi3fxP4F2usdfdsveGAppugHj1IxhoyjY\nOp07x4p1hdIfWVF03RqUrPNXkl72+yh53pv2fUzsdfsd3434GRjGy6J6\notuA/N+xs0+TP2ENlCmDauwO32Okfojvj7CgvsdfdsfRyaFzIGWPdw="} ] script: - "node tests/examples/*.js" ``` The two secure lines in the above example are your TestingBot key and secret, used to run a test on our Grid. To generate these 2 lines, you need to install the Travis CI gem and run these commands with your own key and secret: ```bash travis encrypt username/projectname TESTINGBOT_KEY=my_key travis encrypt username/projectname TESTINGBOT_SECRET=my_secret ``` Now you can use the encrypted `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` environment variables in your tests. Below is a NodeJS example: ```bash npm install -g selenium-webdriver ``` ```javascript const webdriver = require('selenium-webdriver'); const chrome = require('selenium-webdriver/chrome'); async function runSampleTest () { let driver; try { let options = new chrome.Options(); options.set('platformName', 'WIN11'); options.set('browserVersion', 'latest'); options.set('tb:options', { 'key': process.env.TESTINGBOT_KEY, 'secret': process.env.TESTINGBOT_SECRET, 'name': (process.env.TRAVIS_BUILD_NUMBER ? ("Travis Build " + process.env.TRAVIS_BUILD_NUMBER) : "Simple Test") }); driver = new webdriver.Builder() .usingServer("https://hub.testingbot.com/wd/hub") .withCapabilities(options) .build(); await driver.get('https://www.google.com'); var title = await driver.getTitle(); console.log("title is: " + title); } catch (e) { console.log(e); } finally { if (driver) { await driver.quit(); } } } runSampleTest(); ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/accessibility # Accessibility testing Automated Accessibility Testing (A11y) allows you to test if your product follows the Web Content Accessibility Guidelines (WCAG) rules and other guidelines, to ensure your product is accessible to all types of users. TestingBot offers [website accessibility testing](https://testingbot.com/support/accessibility/web), which can be directly integrated in your tests. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/accessibility/web # Accessibility Web Testing Automated Accessibility Testing (A11y) allows you to test if your website follows the Web Content Accessibility Guidelines (WCAG) rules and other guidelines, to ensure your website is accessible to all types of users. Please pick a test automation framework to see documentation on how to perform accessibility testing. [![Selenium](https://testingbot.com/assets/integrations/selenium-1403d7bf83f64f56f05cea3fe50ae45bc3025d5a2433b7b05d2b199fec753cf6.svg) Selenium Accessibility Testing ](https://testingbot.com/support/accessibility/web/selenium) ## Accessibility ruleset TestingBot uses multiple rules to check for accessibility violations. To see an overview of these rules, please consult the [rules for automatic accessibility checks](https://testingbot.com/support/accessibility/web/rules). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/accessibility/web/scheduled # Accessibility Testing Scheduler Web accessibility is a key part of building inclusive applications—but manually testing for WCAG compliance is often tedious, time-consuming, and easy to overlook in fast-paced development cycles. To streamline this process, TestingBot introduces the Accessibility Scheduler: a developer-friendly tool that automates accessibility scans across your URLs. It runs scheduled checks, highlights WCAG violations and generates detailed reports, which will help you catch issues early. With support for **one-time and recurring scans** , this feature reduces manual QA effort and keeps your workflow efficient. It also helps you prepare for upcoming regulations like the **European Accessibility Act** , by ensuring your app meets legal accessibility standards with minimal friction. ## Key Features - **Automated Scheduling:** Configure one-time or recurring accessibility scans with ease. - **WCAG Compliance:** Validate against WCAG levels to ensure your site meets accessibility standards. - **Support for Local and Authenticated Pages:** Run scans in local or staging environments using the [TestingBot Tunnel](https://testingbot.com/support/tunnel). ## Get started ### 1. Add URL To get started, you need to create a new accessibility test. You can do this by going to the [Accessibility Tests](https://testingbot.com/members/accessibility) page and entering your URL in the form. ![Add Accessibility Test](https://testingbot.com/assets/support/accessibility/add_test-7ce020dcfb0d5e76aef0252e9455d5493f5d1c7847bb05fb61dbdd5b89e3b4d5.jpg) ### 2. Schedule Test You will end up on the edit page, where you can specify how frequently you want to run the test. Choose to schedule the test to run once or at regular intervals (daily, weekly, monthly or at a custom interval). ![Schedule Accessibility Test](https://testingbot.com/assets/support/accessibility/schedule-6ddb7ab66592eef9a7aa135d6af9eea58ecf91a6845a08fc1380d8600c3ae568.jpg) ### 3. Alert When a scheduled accessibility test fails, you can choose to be alerted through email, webhook or SMS text message. ![Accessibility Test Alerts](https://testingbot.com/assets/support/accessibility/alert-3a12ae9d85c28bc590319338b0af86b34a2b0d7020c96ada476171ec366a5096.jpg) ## Results The result page will display the results of the test, including the number of violations found, the rules that were violated and the severity of each violation. You can also view the details of each violation, including the affected elements and the recommended fixes. ![Accessibility Test Results](https://testingbot.com/assets/support/accessibility/results-b40abc1b0648a65423db921a65f17c193a3692225a01cba86ce5979021e08abb.jpg) ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/accessibility/web/selenium # Accessibility testing Automated Accessibility Testing (A11y) allows you to test if your website follows the Web Content Accessibility Guidelines (WCAG) rules and other guidelines, to ensure your website is accessible to all types of users. ## Example The following example will start a browser in the TestingBot browser grid, navigate to a website and call the custom `tb:accessibility` command to fetch the accessibility results. We will then retrieve the results from the accessibility test from inside our automated test. The results will be saved in a JSON file for inspection. [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#) [Python](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) ```ruby #!/usr/bin/env ruby require 'rubygems' require 'selenium-webdriver' options = Selenium::WebDriver::Options.chrome options.browser_version = 'latest' options.platform_name = 'WIN11' options.add_option('tb:options', { 'name' => 'Accessibility Test' }) driver = Selenium::WebDriver.for( :remote, url: "https://api_key:api_secret@hub.testingbot.com/wd/hub", capabilities: options ) driver.navigate.to "https://testingbot.com" result = driver.execute_script('tb:accessibility') require 'json' File.write("path/to/report.json", JSON.generate(result)) driver.quit ``` ```java import com.fasterxml.jackson.databind.ObjectMapper; import org.openqa.selenium.*; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.remote.RemoteWebDriver; import java.io.FileWriter; import java.net.URL; import java.nio.file.Files; import java.nio.file.Path; import java.util.HashMap; import java.util.Map; public class AccessibilityTest { public static final String KEY = "api_key"; public static final String SECRET = "api_secret"; public static final String REMOTE_URL = "https://" + KEY + ":" + SECRET + "@hub.testingbot.com/wd/hub"; public static void main(String[] args) { WebDriver driver = null; try { ChromeOptions options = new ChromeOptions(); options.setBrowserVersion("latest"); options.setPlatformName("WIN11"); Map tbOptions = new HashMap<>(); tbOptions.put("name", "Accessibility Test"); options.setCapability("tb:options", tbOptions); driver = new RemoteWebDriver(new URL(REMOTE_URL), options); driver.get("https://testingbot.com"); JavascriptExecutor js = (JavascriptExecutor) driver; Object result = js.executeScript("tb:accessibility"); Path outputPath = Files.createTempFile("accessibility-report", ".json"); try (FileWriter writer = new FileWriter(outputPath.toFile())) { writer.write(new ObjectMapper().writeValueAsString(result)); System.out.println("Accessibility report saved to: " + outputPath.toAbsolutePath()); } } catch (Exception e) { e.printStackTrace(); } finally { if (driver != null) { driver.quit(); } } } } ``` ```php setCapability('platformName', 'WIN11'); $capabilities->setCapability('browserVersion', 'latest'); $capabilities->setCapability('tb:options', ['name' => 'Accessibility Test']); $web_driver = RemoteWebDriver::create( "https://api_key:api_secret@hub.testingbot.com/wd/hub", $capabilities, 120000 ); $web_driver->get("https://testingbot.com"); $result = $web_driver->executeScript('tb:accessibility'); file_put_contents("path/to/report.json", json_encode($result, JSON_PRETTY_PRINT)); $web_driver->quit(); ?> ``` ```python import json from selenium import webdriver from selenium.webdriver.chrome.options import Options options = Options() options.browser_version = 'latest' options.platform_name = 'WIN11' options.set_capability('tb:options', {'name': 'Accessibility Test'}) driver = webdriver.Remote( command_executor='https://api_key:api_secret@hub.testingbot.com/wd/hub', options=options ) driver.get("https://testingbot.com") result = driver.execute_script('tb:accessibility') with open("path/to/report.json", "w") as file: json.dump(result, file, indent=2) driver.quit() ``` ```javascript const { Builder } = require('selenium-webdriver'); const fs = require('fs/promises'); const TESTINGBOT_KEY = 'api_key'; const TESTINGBOT_SECRET = 'api_secret'; const capabilities = { browserName: 'chrome', browserVersion: 'latest', platformName: 'WIN11', 'tb:options': { name: 'Accessibility Test' } }; async function runAccessibilityTest() { let driver; try { driver = await new Builder() .usingServer(`https://${TESTINGBOT_KEY}:${TESTINGBOT_SECRET}@hub.testingbot.com/wd/hub`) .withCapabilities(capabilities) .build(); await driver.get('https://testingbot.com'); const result = await driver.executeScript('tb:accessibility'); const filePath = 'accessibility-report.json'; await fs.writeFile(filePath, JSON.stringify(result, null, 2)); console.log(`Accessibility report saved to ${filePath}`); } catch (error) { console.error('Test failed:', error); } finally { if (driver) { await driver.quit(); } } } runAccessibilityTest(); ``` ```csharp using System; using System.Collections.Generic; using System.IO; using Newtonsoft.Json; using OpenQA.Selenium; using OpenQA.Selenium.Chrome; using OpenQA.Selenium.Remote; namespace SeleniumTest { class Program { static readonly string report = @"path/to/report.json"; static void Main(string[] args) { var options = new ChromeOptions(); options.BrowserVersion = "latest"; options.PlatformName = "WIN11"; options.AddAdditionalOption("tb:options", new Dictionary { { "name", "Accessibility Test" } }); var driver = new RemoteWebDriver( new Uri("https://api_key:api_secret@hub.testingbot.com/wd/hub"), options ); driver.Navigate().GoToUrl("https://testingbot.com"); var result = driver.ExecuteScript("tb:accessibility"); File.WriteAllText(report, JsonConvert.SerializeObject(result, Formatting.Indented)); driver.Quit(); } } } ``` You could write `assertions` to pass/fail your test depending on the accessibility issues found. ## Results After running the example test, you will find a `report.json` file with the results of the accessibility test. The file will contain the following structure: ```json { "testEngine": { }, "testRunner": { }, "testEnvironment": { }, "toolOptions": { }, "url": "https://testingbot.com", "timestamp": "2026-05-13T12:00:00.000Z", "passes": [], "violations": [], "incomplete": [], "inapplicable": [] } ``` - `violations` includes the rules that were violated and need to be fixed. - `passes` the rules that have passed successfully. - `incomplete` incomplete results, either due to an error or aborted test - `inapplicable` no matching content was found for these rules. An overview of all accessibility rules that will be checked can be found on [A11Y rules overview](https://testingbot.com/support/accessibility/web/rules). ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/accessibility/web/rules # Accessibility Web Testing Rules Below is an overview of all the A11Y rules that TestingBot uses during automated web accessibility testing. Click a rule to see an explanation of the check and how to fix potential problems. ## Accessibility ruleset WCAG 2.0 Level A & AA Rules | Rule ID | Description | | --- | --- | | [area-alt](https://testingbot.com/support/accessibility/web/rules/area-alt) | Ensure \ elements of image maps have alternative text | | [aria-allowed-attr](https://testingbot.com/support/accessibility/web/rules/aria-allowed-attr) | Ensure an element's role supports its ARIA attributes | | [aria-braille-equivalent](https://testingbot.com/support/accessibility/web/rules/aria-braille-equivalent) | Make sure aria-braillelabel and aria-brailleroledescription have a non-braille equivalent | | [aria-command-name](https://testingbot.com/support/accessibility/web/rules/aria-command-name) | ARIA commands must have an accessible name | | [aria-conditional-attr](https://testingbot.com/support/accessibility/web/rules/aria-conditional-attr) | Ensure ARIA attributes are used as described in the specification of the element's role | | [aria-deprecated-role](https://testingbot.com/support/accessibility/web/rules/aria-deprecated-role) | Make sure elements do not use deprecated roles | | [aria-hidden-body](https://testingbot.com/support/accessibility/web/rules/aria-hidden-body) | Make sure aria-hidden="true" is not present on the document body. | | [aria-hidden-focus](https://testingbot.com/support/accessibility/web/rules/aria-hidden-focus) | Ensure aria-hidden elements are not focusable nor contain focusable elements | | [aria-input-field-name](https://testingbot.com/support/accessibility/web/rules/aria-input-field-name) | Ensure every ARIA input field has an accessible name | | [aria-meter-name](https://testingbot.com/support/accessibility/web/rules/aria-meter-name) | Make sure every ARIA meter node has an accessible name | | [aria-progressbar-name](https://testingbot.com/support/accessibility/web/rules/aria-progressbar-name) | Every ARIA progressbar node must have an accessible name | | [aria-prohibited-attr](https://testingbot.com/support/accessibility/web/rules/aria-prohibited-attr) | Ensure ARIA attributes are not prohibited for an element's role | | [aria-required-attr](https://testingbot.com/support/accessibility/web/rules/aria-required-attr) | Make sure elements with ARIA roles have all required ARIA attributes | | [aria-required-children](https://testingbot.com/support/accessibility/web/rules/aria-required-children) | Ensure elements with an ARIA role that require child roles contain them | | [aria-required-parent](https://testingbot.com/support/accessibility/web/rules/aria-required-parent) | Make sure elements with an ARIA role that require parent roles are contained by them | | [aria-roles](https://testingbot.com/support/accessibility/web/rules/aria-roles) | Ensure all elements with a role attribute use a valid value | | [aria-toggle-field-name](https://testingbot.com/support/accessibility/web/rules/aria-toggle-field-name) | Make sure every ARIA toggle field has an accessible name | | [aria-tooltip-name](https://testingbot.com/support/accessibility/web/rules/aria-tooltip-name) | Ensure every ARIA tooltip node has an accessible name | | [aria-valid-attr-value](https://testingbot.com/support/accessibility/web/rules/aria-valid-attr-value) | Make sure all ARIA attributes have valid values | | [aria-valid-attr](https://testingbot.com/support/accessibility/web/rules/aria-valid-attr) | Ensure attributes that begin with aria- are valid ARIA attributes | | [blink](https://testingbot.com/support/accessibility/web/rules/blink) | Make sure \ elements are not used | | [button-name](https://testingbot.com/support/accessibility/web/rules/button-name) | Ensure buttons have discernible text | | [bypass](https://testingbot.com/support/accessibility/web/rules/bypass) | Ensure each page has at least one mechanism for a user to bypass navigation and jump straight to the content | | [color-contrast](https://testingbot.com/support/accessibility/web/rules/color-contrast) | Make sure each page has at least one mechanism for a user to bypass navigation and jump straight to the content | | [definition-list](https://testingbot.com/support/accessibility/web/rules/definition-list) | Ensure \ elements are structured correctly | | [dlitem](https://testingbot.com/support/accessibility/web/rules/dlitem) | Ensure \ and \ elements are contained by a \ | | [document-title](https://testingbot.com/support/accessibility/web/rules/document-title) | Make sure each HTML document contains a non-empty \ element | | [duplicate-id-aria](https://testingbot.com/support/accessibility/web/rules/duplicate-id-aria) | Ensure every id attribute value used in ARIA and in labels is unique | | [form-field-multiple-labels](https://testingbot.com/support/accessibility/web/rules/form-field-multiple-labels) | Ensure form field does not have multiple label elements | | [frame-focusable-content](https://testingbot.com/support/accessibility/web/rules/frame-focusable-content) | Ensure \ and \ elements with focusable content do not have tabindex=-1 | | [frame-title-unique](https://testingbot.com/support/accessibility/web/rules/frame-title-unique) | Ensure \ and \ elements contain a unique title attribute | | [frame-title](https://testingbot.com/support/accessibility/web/rules/frame-title) | Make sure \ and \ elements have an accessible name | | [html-has-lang](https://testingbot.com/support/accessibility/web/rules/html-has-lang) | Ensure every HTML document has a lang attribute | | [html-lang-valid](https://testingbot.com/support/accessibility/web/rules/html-lang-valid) | Ensure the lang attribute of the \ element has a valid value | | [html-xml-lang-mismatch](https://testingbot.com/support/accessibility/web/rules/html-xml-lang-mismatch) | Ensure that HTML elements with both valid lang and xml:lang attributes agree on the base language of the page | | [image-alt](https://testingbot.com/support/accessibility/web/rules/image-alt) | Ensure \ elements have alternative text or a role of none or presentation | | [input-button-name](https://testingbot.com/support/accessibility/web/rules/input-button-name) | Ensure input buttons have discernible text | | [input-image-alt](https://testingbot.com/support/accessibility/web/rules/input-image-alt) | Ensure \ elements have alternative text | | [label](https://testingbot.com/support/accessibility/web/rules/label) | Ensure every form element has a label | | [link-in-text-block](https://testingbot.com/support/accessibility/web/rules/link-in-text-block) | Ensure links are distinguished from surrounding text in a way that does not rely on color | | [link-name](https://testingbot.com/support/accessibility/web/rules/link-name) | Ensure links have discernible text | | [list](https://testingbot.com/support/accessibility/web/rules/list) | Ensure that lists are structured correctly | | [listitem](https://testingbot.com/support/accessibility/web/rules/listitem) | Ensure \ elements are used semantically | | [marquee](https://testingbot.com/support/accessibility/web/rules/marquee) | Ensure \ elements are not used | | [meta-refresh](https://testingbot.com/support/accessibility/web/rules/meta-refresh) | Ensure \ is not used for delayed refresh | | [meta-viewport](https://testingbot.com/support/accessibility/web/rules/meta-viewport) | Ensure \ does not disable text scaling and zooming | | [nested-interactive](https://testingbot.com/support/accessibility/web/rules/nested-interactive) | Ensure interactive controls are not nested as they are not always announced by screen readers or can cause focus problems for assistive technologies | | [no-autoplay-audio](https://testingbot.com/support/accessibility/web/rules/no-autoplay-audio) | Ensure \ or \ elements do not autoplay audio for more than 3 seconds without a control mechanism to stop or mute the audio | | [object-alt](https://testingbot.com/support/accessibility/web/rules/object-alt) | Ensure \ elements have alternative text | | [role-img-alt](https://testingbot.com/support/accessibility/web/rules/role-img-alt) | Ensure [role="img"] elements have alternative text | | [scrollable-region-focusable](https://testingbot.com/support/accessibility/web/rules/scrollable-region-focusable) | Ensure elements that have scrollable content are accessible by keyboard | | [select-name](https://testingbot.com/support/accessibility/web/rules/select-name) | Ensure select element has an accessible name | | [server-side-image-map](https://testingbot.com/support/accessibility/web/rules/server-side-image-map) | Ensure that server-side image maps are not used | | [summary-name](https://testingbot.com/support/accessibility/web/rules/summary-name) | Ensure summary elements have discernible text | | [svg-img-alt](https://testingbot.com/support/accessibility/web/rules/svg-img-alt) | Ensure \ elements with an img, graphics-document or graphics-symbol role have an accessible text | | [td-headers-attr](https://testingbot.com/support/accessibility/web/rules/td-headers-attr) | Ensure that each cell in a table that uses the headers attribute refers only to other \ elements in that table | | [th-has-data-cells](https://testingbot.com/support/accessibility/web/rules/th-has-data-cells) | Ensure that \ elements and elements with role=columnheader/rowheader have data cells they describe | | [valid-lang](https://testingbot.com/support/accessibility/web/rules/valid-lang) | Ensure lang attributes have valid values | | [video-caption](https://testingbot.com/support/accessibility/web/rules/video-caption) | Ensure \ elements have captions | ## Accessibility ruleset WCAG 2.1 Level A & AA Rules | Rule ID | Description | | --- | --- | | [autocomplete-valid](https://testingbot.com/support/accessibility/web/rules/autocomplete-valid) | Ensure the autocomplete attribute is correct and suitable for the form field | | [avoid-inline-spacing](https://testingbot.com/support/accessibility/web/rules/avoid-inline-spacing) | Ensure that text spacing set through style attributes can be adjusted with custom stylesheets | ## Accessibility ruleset WCAG 2.2 Level A & AA Rules | Rule ID | Description | | --- | --- | | [target-size](https://testingbot.com/support/accessibility/web/rules/target-size) | Ensure touch targets have sufficient size and space | ## Best Practices | Rule ID | Description | | --- | --- | | [accesskeys](https://testingbot.com/support/accessibility/web/rules/accesskeys) | Ensure every accesskey attribute value is unique | | [aria-allowed-role](https://testingbot.com/support/accessibility/web/rules/aria-allowed-role) | Ensure role attribute has an appropriate value for the element | | [aria-dialog-name](https://testingbot.com/support/accessibility/web/rules/aria-dialog-name) | Ensure every ARIA dialog and alertdialog node has an accessible name | | [aria-text](https://testingbot.com/support/accessibility/web/rules/aria-text) | Make sure role="text" is used on elements with no focusable descendants | | [aria-treeitem-name](https://testingbot.com/support/accessibility/web/rules/aria-treeitem-name) | Make sure every ARIA treeitem node has an accessible name | | [empty-heading](https://testingbot.com/support/accessibility/web/rules/empty-heading) | Make sure headings have discernible text | | [empty-table-header](https://testingbot.com/support/accessibility/web/rules/empty-table-header) | Ensure table headers have discernible text | | [heading-order](https://testingbot.com/support/accessibility/web/rules/heading-order) | Ensure the order of headings is semantically correct | | [image-redundant-alt](https://testingbot.com/support/accessibility/web/rules/image-redundant-alt) | Ensure image alternative is not repeated as text | | [label-title-only](https://testingbot.com/support/accessibility/web/rules/label-title-only) | Make sure that every form element has a visible label and is not solely labeled using hidden labels, or the title or aria-describedby attributes | | [landmark-banner-is-top-level](https://testingbot.com/support/accessibility/web/rules/landmark-banner-is-top-level) | Ensure the banner landmark is at top level | | [landmark-complementary-is-top-level](https://testingbot.com/support/accessibility/web/rules/landmark-complementary-is-top-level) | Make sure the complementary landmark or aside is at top level | | [landmark-contentinfo-is-top-level](https://testingbot.com/support/accessibility/web/rules/landmark-contentinfo-is-top-level) | Make sure the contentinfo landmark is at top level | | [landmark-main-is-top-level](https://testingbot.com/support/accessibility/web/rules/landmark-main-is-top-level) | Ensure the main landmark is at top level | | [landmark-no-duplicate-banner](https://testingbot.com/support/accessibility/web/rules/landmark-no-duplicate-banner) | Make sure the document has at most one banner landmark | | [landmark-no-duplicate-contentinfo](https://testingbot.com/support/accessibility/web/rules/landmark-no-duplicate-contentinfo) | Ensure the document has at most one contentinfo landmark | | [landmark-no-duplicate-main](https://testingbot.com/support/accessibility/web/rules/landmark-no-duplicate-main) | Make sure the document has at most one main landmark | | [landmark-one-main](https://testingbot.com/support/accessibility/web/rules/landmark-one-main) | Ensure the document has a main landmark | | [landmark-unique](https://testingbot.com/support/accessibility/web/rules/landmark-unique) | Ensure landmarks are unique | | [meta-viewport-large](https://testingbot.com/support/accessibility/web/rules/meta-viewport-large) | Ensure \ can scale a significant amount | | [page-has-heading-one](https://testingbot.com/support/accessibility/web/rules/page-has-heading-one) | Ensure that the page, or at least one of its frames contains a level-one heading | | [presentation-role-conflict](https://testingbot.com/support/accessibility/web/rules/presentation-role-conflict) | Ensure elements marked as presentational do not have global ARIA or tabindex so that all screen readers ignore them | | [region](https://testingbot.com/support/accessibility/web/rules/region) | Ensure all page content is contained by landmarks | | [scope-attr-valid](https://testingbot.com/support/accessibility/web/rules/scope-attr-valid) | Make sure the scope attribute is used correctly on tables | | [skip-link](https://testingbot.com/support/accessibility/web/rules/skip-link) | Ensure all skip links have a focusable target | | [tabindex](https://testingbot.com/support/accessibility/web/rules/tabindex) | Make sure tabindex attribute values are not greater than 0 | | [table-duplicate-name](https://testingbot.com/support/accessibility/web/rules/table-duplicate-name) | Make sure the \ element does not contain the same text as the summary attribute | | [color-contrast-enhanced](https://testingbot.com/support/accessibility/web/rules/color-contrast-enhanced) | Ensure the contrast between foreground and background colors meets WCAG 2 AAA enhanced contrast ratio thresholds | | [identical-links-same-purpose](https://testingbot.com/support/accessibility/web/rules/identical-links-same-purpose) | Ensure that links with the same accessible name serve a similar purpose | | [meta-refresh-no-exceptions](https://testingbot.com/support/accessibility/web/rules/meta-refresh-no-exceptions) | Ensure \ is not used for delayed refresh | ## Experimental Rules | Rule ID | Description | | --- | --- | | [css-orientation-lock](https://testingbot.com/support/accessibility/web/rules/css-orientation-lock) | Make sure content is not locked to any specific display orientation, and the content is operable in all display orientations | | [focus-order-semantics](https://testingbot.com/support/accessibility/web/rules/focus-order-semantics) | Ensure elements in the focus order have a role appropriate for interactive content | | [hidden-content](https://testingbot.com/support/accessibility/web/rules/hidden-content) | Inform users about hidden content. | | [label-content-name-mismatch](https://testingbot.com/support/accessibility/web/rules/label-content-name-mismatch) | Ensure that elements labelled through their content must have their visible text as part of their accessible name | | [p-as-heading](https://testingbot.com/support/accessibility/web/rules/p-as-heading) | Make sure bold, italic text and font-size is not used to style \ elements as a heading | | [table-fake-caption](https://testingbot.com/support/accessibility/web/rules/table-fake-caption) | Ensure that tables with a caption use the \ element. | | [td-has-header](https://testingbot.com/support/accessibility/web/rules/td-has-header) | Ensure that each non-empty data cell in a \ larger than 3 by 3 has one or more table headers | ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/functions # TestingBot Functions With functions you can perform various actions on the TestingBot browsers, without writing any code. Please see the list of currently available functions below: - [Take screenshots](https://testingbot.com/support/functions/screenshot) - [Generate PDFs](https://testingbot.com/support/functions/pdf) - [Scrape Webpages](https://testingbot.com/support/functions/scrape) - [Serverless Functions](https://testingbot.com/support/functions/serverless) ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/functions/screenshot # Screenshot Function This function will take a screenshot of the page you specified and return the image in `.png` or `.jpg` format. You can take one or more screenshots, in parallel, on different browsers and operating systems without having to write any code. Simply use the HTTP requests listed below in your app to generate a live screenshot of any page, on a real TestingBot browser. ## Example ```bash curl -X POST https://cloud.testingbot.com/screenshot?key=YOUR_KEY&secret=YOUR_SECRET&browserName=chrome&version=latest&platform=WIN11 \ -H 'Content-Type: application/json' \ -d '{"url":"https://testingbot.com"}' > screenshot.png ``` This simple example will run a Puppeteer script on our service and do the following: - Start a Headless Browser (latest version) on our cloud - Connect with Puppeteer to the Browser and navigate to the URL you specified - Take a screenshot of the page - Return the `PNG data` back to you - The `PNG data` is saved in a file called `screenshot.png` ## Specifying browser and version See the example below to specify on which platform configuration you'd like to take your screenshot on. You can specify a `browserName`, `version` and `platform`. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Screenshot Options You can specify additional options with this function call to customize the screenshot taking. This function call can use any of the Puppeteer [page.screenshot](https://pptr.dev/api/puppeteer.screenshotoptions) options. ```bash curl -X POST https://cloud.testingbot.com/screenshot?key=YOUR_KEY&secret=YOUR_SECRET&browserName=chrome&version=latest&platform=WIN11 \ -H 'Content-Type: application/json' \ -d '{"url":"https://testingbot.com", "options":{"fullPage":true}}' > screenshot.png ``` ### Authenticate options You can specify the [page.authenticate](https://pptr.dev/api/puppeteer.page.authenticate) credentials. ```bash curl -X POST https://cloud.testingbot.com/screenshot?key=YOUR_KEY&secret=YOUR_SECRET&browserName=chrome&version=latest&platform=WIN11 \ -H 'Content-Type: application/json' \ -d '{"url":"https://testingbot.com", "authenticate":{ "username": "user", "password": "passwd" }}' > screenshot.png ``` ### Goto options You can specify the [page.goto](https://pptr.dev/api/puppeteer.page.goto) options and add `timeout` and `waitUntil` settings. ```bash curl -X POST https://cloud.testingbot.com/screenshot?key=YOUR_KEY&secret=YOUR_SECRET&browserName=chrome&version=latest&platform=WIN11 \ -H 'Content-Type: application/json' \ -d '{"url":"https://testingbot.com", "gotoOptions":{ "waitUntil": "networkidle2" }}' > screenshot.png ``` ### Extra Headers You can specify the [page.setExtraHTTPHeaders](https://pptr.dev/api/puppeteer.page.setextrahttpheaders) options to add extra HTTP headers to the request our Headless Browser makes. ```bash curl -X POST https://cloud.testingbot.com/screenshot?key=YOUR_KEY&secret=YOUR_SECRET&browserName=chrome&version=latest&platform=WIN11 \ -H 'Content-Type: application/json' \ -d '{"url":"https://testingbot.com", "extraHeaders":{ "foo": "bar" }}' > screenshot.png ``` ### Disable Javascript You can disable Javascript with the [page.setJavaScriptEnabled](https://pptr.dev/api/puppeteer.page.setjavascriptenabled) option. ```bash curl -X POST https://cloud.testingbot.com/screenshot?key=YOUR_KEY&secret=YOUR_SECRET&browserName=chrome&version=latest&platform=WIN11 \ -H 'Content-Type: application/json' \ -d '{"url":"https://testingbot.com", "javascriptEnabled":false}' > screenshot.png ``` ### Add Tags - You can add additional `style tags` with the [page.addStyleTag](https://pptr.dev/api/puppeteer.page.addstyletag) option. - You can add additional `script tags` with the [page.addScriptTag](https://pptr.dev/api/puppeteer.page.addscripttag) option. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/functions/pdf # PDF Function This function will generate a PDF from any webpage. TestingBot will launch a remote browser, visit the URL you specified and take a PDF from it. Use the URLs below to take an instant PDF from a page, without having to write any code. Instead of relying on tools such as wkhtmltopdf, PDFShift, Athena, PDFBlade or others, you can rely on the power of Chrome's rendering engine to take a high quality PDF through a Function (which uses Puppeteer behind the scenes). ## Example ```bash curl -X POST https://cloud.testingbot.com/pdf?key=YOUR_KEY&secret=YOUR_SECRET&browserName=chrome&version=latest&platform=WIN11 \ -H 'Content-Type: application/json' \ -d '{"url":"https://testingbot.com"}' > test.pdf ``` This simple example will run a Puppeteer script on our service and do the following: - Start a Headless Browser (latest version) on our cloud - Connect with Puppeteer to the Browser and navigate to the URL you specified - Generate a PDF from the page - Return the `PDF file` back to you, in a file called `test.pdf` ## Specifying browser and version See the example below to specify on which platform configuration you'd like to generate the PDF on. You can specify a `browserName`, `version` and `platform`. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## PDF Options You can specify additional options with this function call to customize the PDF taking. This function call can use any of the Puppeteer [puppeteer.pdfoptions](https://pptr.dev/api/puppeteer.pdfoptions) options. ```bash curl -X POST https://cloud.testingbot.com/pdf?key=YOUR_KEY&secret=YOUR_SECRET&browserName=chrome&version=latest&platform=WIN11 \ -H 'Content-Type: application/json' \ -d '{"url":"https://testingbot.com", "options":{"fullPage":true}}' > test.pdf ``` ### Authenticate options You can specify the [page.authenticate](https://pptr.dev/api/puppeteer.page.authenticate) credentials. ```bash curl -X POST https://cloud.testingbot.com/pdf?key=YOUR_KEY&secret=YOUR_SECRET&browserName=chrome&version=latest&platform=WIN11 \ -H 'Content-Type: application/json' \ -d '{"url":"https://testingbot.com", "authenticate":{ "username": "user", "password": "passwd" }}' > test.pdf ``` ### Goto options You can specify the [page.goto](https://pptr.dev/api/puppeteer.page.goto) options and add `timeout` and `waitUntil` settings. ```bash curl -X POST https://cloud.testingbot.com/pdf?key=YOUR_KEY&secret=YOUR_SECRET&browserName=chrome&version=latest&platform=WIN11 \ -H 'Content-Type: application/json' \ -d '{"url":"https://testingbot.com", "gotoOptions":{ "waitUntil": "networkidle2" }}' > test.pdf ``` ### Extra Headers You can specify the [page.setExtraHTTPHeaders](https://pptr.dev/api/puppeteer.page.setextrahttpheaders) options to add extra HTTP headers to the request that the TestingBot browser makes. ```bash curl -X POST https://cloud.testingbot.com/pdf?key=YOUR_KEY&secret=YOUR_SECRET&browserName=chrome&version=latest&platform=WIN11 \ -H 'Content-Type: application/json' \ -d '{"url":"https://testingbot.com", "extraHeaders":{ "foo": "bar" }}' > test.pdf ``` ### Disable Javascript You can disable Javascript with the [page.setJavaScriptEnabled](https://pptr.dev/api/puppeteer.page.setjavascriptenabled) option. ```bash curl -X POST https://cloud.testingbot.com/pdf?key=YOUR_KEY&secret=YOUR_SECRET&browserName=chrome&version=latest&platform=WIN11 \ -H 'Content-Type: application/json' \ -d '{"url":"https://testingbot.com", "javascriptEnabled":false}' > test.pdf ``` ### Emulate Media Change the CSS media type of the page with [page.emulateMediaType](https://pptr.dev/api/puppeteer.page.emulatemediatype). ```bash curl -X POST https://cloud.testingbot.com/pdf?key=YOUR_KEY&secret=YOUR_SECRET&browserName=chrome&version=latest&platform=WIN11 \ -H 'Content-Type: application/json' \ -d '{"url":"https://testingbot.com", "emulateMedia":"print"}' > test.pdf ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/functions/scrape # Scrape Function With the Scrape function you can visit any webpage on the internet with a remote, real browser running on TestingBot. You can fetch specific elements from the page and TestingBot will return it as a structured JSON response. By default, TestingBot will navigate to the URL and wait for all content to be loaded. TestingBot will then wait up to 30 seconds for an element to appear, assuming it is not yet in the DOM. ## Structured data The Scrape function can return structured data for several specific use cases. See the list below on what we currently support: - [Amazon Products Scraper API](https://testingbot.com/support/functions/scrape/amazon-products) ## Example ```bash curl -X POST https://cloud.testingbot.com/scrape?key=YOUR_KEY&secret=YOUR_SECRET&browserName=chrome&version=latest&platform=WIN11 \ -H 'Content-Type: application/json' \ -d '{"url":"https://testingbot.com", "elements":[{"selector":"body h2"}]}' ``` This simple example will run a Puppeteer script on our service and do the following: - Start a Headless Browser (latest version) on our cloud - Connect with Puppeteer to the Browser and navigate to the URL you specified - Look for any elements in the DOM that match the selector `body h2` - Return a JSON response with the results that are found, see the example response below. ```json [{"selector":"body h2","results":[{"html":"Automated Testing","text":"","width":340,"height":84,"top":1852.40625,"left":780,"attributes":[]},{"html":"Live Testing","text":"","width":150,"height":84,"top":2727.890625,"left":-146.71875,"attributes":[]},{"html":"+4800 Browsers & Devices","text":"","width":184,"height":168,"top":3455.265625,"left":780,"attributes":[]},{"html":"Integrate TestingBot into your setup","text":"Integrate TestingBot into your setup","width":549,"height":35,"top":2097.9375,"left":117.03125,"attributes":[]}]}] ``` ## Specifying browser and version See the example below to specify on which platform configuration you'd like to scrape a webpage on. You can specify a `browserName`, `version` and `platform`. ![OS selected](https://testingbot.com/assets/environments/svg/windows11-0e1b28bc0fdd5034d3e4d3dc8d346c500a8c6522facf4b45d0da56537c1f1c6d.svg) Windows 11 › ![Browser selected](https://testingbot.com/assets/environments/svg/chrome-c4081ff447d2d898d4afcb8f074a907c960e6f007716c1a1d119eee6803c4042.svg) Chrome 139 Loading environments... Please wait while we load the available browsers and platforms. ## Scrape Options You can specify additional options to use while using the scrape functionality. ### Authenticate options You can specify the [page.authenticate](https://pptr.dev/api/puppeteer.page.authenticate) credentials. ```bash curl -X POST https://cloud.testingbot.com/scrape?key=YOUR_KEY&secret=YOUR_SECRET&browserName=chrome&version=latest&platform=WIN11 \ -H 'Content-Type: application/json' \ -d '{"url":"https://testingbot.com", "authenticate":{ "username": "user", "password": "passwd" }, "elements":[{"selector":"body h2"}]}' ``` ### Goto options You can specify the [page.goto](https://pptr.dev/api/puppeteer.page.goto) options and add `timeout` and `waitUntil` settings. ```bash curl -X POST https://cloud.testingbot.com/scrape?key=YOUR_KEY&secret=YOUR_SECRET&browserName=chrome&version=latest&platform=WIN11 \ -H 'Content-Type: application/json' \ -d '{"url":"https://testingbot.com", "gotoOptions":{ "waitUntil": "networkidle2" }, "elements":[{"selector":"body h2"}]}' ``` ### Extra Headers You can specify the [page.setExtraHTTPHeaders](https://pptr.dev/api/puppeteer.page.setextrahttpheaders) options to add extra HTTP headers to the request that the TestingBot browser makes. ```bash curl -X POST https://cloud.testingbot.com/scrape?key=YOUR_KEY&secret=YOUR_SECRET&browserName=chrome&version=latest&platform=WIN11 \ -H 'Content-Type: application/json' \ -d '{"url":"https://testingbot.com", "extraHeaders":{ "foo": "bar" }, "elements":[{"selector":"body h2"}]}' ``` ### Disable Javascript You can disable Javascript with the [page.setJavaScriptEnabled](https://pptr.dev/api/puppeteer.page.setjavascriptenabled) option. ```bash curl -X POST https://cloud.testingbot.com/scrape?key=YOUR_KEY&secret=YOUR_SECRET&browserName=chrome&version=latest&platform=WIN11 \ -H 'Content-Type: application/json' \ -d '{"url":"https://testingbot.com", "javascriptEnabled":false, "elements":[{"selector":"body h2"}]}' ``` ### Emulate Media Change the CSS media type of the page with [page.emulateMediaType](https://pptr.dev/api/puppeteer.page.emulatemediatype). ```bash curl -X POST https://cloud.testingbot.com/scrape?key=YOUR_KEY&secret=YOUR_SECRET&browserName=chrome&version=latest&platform=WIN11 \ -H 'Content-Type: application/json' \ -d '{"url":"https://testingbot.com", "emulateMedia":"print", "elements":[{"selector":"body h2"}]}' ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/functions/scrape/amazon-products # Scraping Amazon Products Stay on top of your competitors products and pricing strategies by monitoring millions of Amazon product listings, with a real remote browser. Fetch detailed information about any Amazon product, including title, price, description, images, reviews and more. TestingBot provides a scrape function which you can call with an Amazon product ASIN, and it will return a structured JSON response with all the details of the product. ## Example ```bash curl -X POST https://cloud.testingbot.com/scrape/amazon-product?key=YOUR_KEY&secret=YOUR_SECRET \ -H 'Content-Type: application/json' \ -d '{"asin":"{AMAZON_ASIN}"}' ``` Replace `{AMAZON_ASIN}` with the actual ASIN of the product you want to scrape. The response will contain all the details of the product, including title, price, description, images, reviews and more. ```json { "name": "Amazon Product Name", "product_information": { "product_dimensions": "8.1 x 8.1 x 0.04 inches", "item_weight": "3.52 ounces", "asin": "ACD3L74ZSQ", "item_model_number": "AV192", "customer_reviews": { "ratings_count": 14, "stars": 4.2 }, "best_sellers_rank": [ "#308,205 in Home & Kitchen (See Top 100 in Home & Kitchen)" ], "is_discontinued_by_manufacturer": null, "date_first_available": null, "manufacturer": "TB,LTD", "country_of_origin": null }, "brand": "Brand: TB", "brand_url": "/TB", "full_description": "The description", "pricing": "$12.98", "shipping_price": "Delivery Friday, August 22", "availability_quantity": null, "availability_status": "In Stock", "is_coupon_exists": true, "images": [ "https://m.media-amazon.com/images/I/82Dz8V7C3CL._AC_SX425_.jpg" ], "product_category": "Home & Kitchen", "average_rating": 4.2, "feature_bullets": [ "A cool scraper from TestingBot." ], "total_reviews": 14, "model": "AD195", "aplus_present": true } ``` ## Geolocations Some Amazon products are only available in specific countries, or the shipping price may vary based on the country. You can specify a `geoCountryCode` to scrape the product from a specific country. ```bash curl -X POST https://cloud.testingbot.com/scrape/amazon-product?key=YOUR_KEY&secret=YOUR_SECRET \ -H 'Content-Type: application/json' \ -d '{"asin":"{AMAZON_ASIN}", "geoCountryCode":"US"}' ``` ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/support/functions/serverless # /function This function endpoint allows you to `POST` Javascript code. TestingBot will run the Javascript code on the browser you specified and return the response generated from the code. ## Basic Example In the example below, we'll pass code that directly interfaces with Puppeteer. ```bash curl -X POST https://cloud.testingbot.com/scrape?key=YOUR_KEY&secret=YOUR_SECRET \ -H 'Content-Type: application/json' \ -d '{ "code": "module.exports=async({page:a,context:b})=>{const{url:c}=b;await a.goto(c);const d=await a.content();return{data:d,type:\\\"application/html\\\"}};", "context": { "url": "https://testingbot.com/" } }' ``` We use the `page` object, which is a [Puppeteer Page Object](https://pptr.dev/api/puppeteer.page) and `context` which you can compare with function arguments that we pass in as an object. The example above will open a new browser session and execute the code in the `code` with `context` as arguments. Once the function has completed, you will receive the response. ## Advanced Example You can run other Javascript code in the serverless function as well, for example let's say you want to calculate something. Of course this is a trivial example, but it shows the capabilities of running arbitrary Javascript code in the cloud. ```bash curl -X POST https://cloud.testingbot.com/scrape?key=YOUR_KEY&secret=YOUR_SECRET \ -H 'Content-Type: application/json' \ -d '{ "code": "module.exports=async()=>{const d = 4+4; return{data:d,type:\"application/html\"}};", "context": {} }' ``` This will run a function that simply does a `4 + 4` calculation and return the result. ## Looking for more help? Have questions or need more information? Reach out via email or Slack. [Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw) --- URL: https://testingbot.com/browserstack-alternative TestingBot vs BrowserStack # The affordable BrowserStack alternative Real desktop browsers, physical iPhones and Android phones, visual, accessibility and AI testing. One transparent plan. GDPR hosted in the EU. Already on BrowserStack? Send us a recent invoice and we'll knock **10% off** your TestingBot subscription for the switch. [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) - **Tests executed:** 1+ bn - **Active users:** 180k - **Browsers & devices:** 6100+ Switching BrowserStack TestingBot Save 71% $125/mo saved 1 day to switch GDPR · EU hosting Trusted by some of the world's most innovative companies The shortlist ## Six cross-browser and device testing platforms, compared Prices and capabilities below are taken from each vendor's own pricing pages, checked on 21 August 2026. Where a vendor does not publish something we say so rather than guessing. TestingBot, BrowserStack, Sauce Labs, TestMu AI, TestGrid and Kobiton compared on pricing, metering, device coverage, hosting and free trials.| Capability | TestingBot | BrowserStack | [Sauce Labs](https://testingbot.com/saucelabs-alternative) | [TestMu AI](https://testingbot.com/lambdatest-alternative) | TestGrid | Kobiton | | --- | --- | --- | --- | --- | --- | --- | | Cheapest plan that includes real device automation Annual billing. TestingBot publishes in euro, the others in US dollars, so these are list prices and not a currency conversion. | EUR 50 Automated, 1 parallel | $175 Automate Desktop & Mobile, per parallel | $199 Real Device Cloud | $199 Real Device Plus Automation, per parallel | $199 Starter, per seat, 4 devices or browsers | From $83 Startup, 500 minutes per month | | How usage is metered | Parallel sessions | Parallel sessions | Parallel sessions | Parallel sessions | Seats plus AI tokens | Minutes per month | | Emulators and simulators alongside real devices BrowserStack publishes no emulator or simulator option in App Live or App Automate. TestGrid and Kobiton publish real devices and real browsers only, and name no emulator or simulator product. | | | 1700+ published | | Not published | Not published | | EU hosting or data residency TestingBot runs every plan from the EU and Switzerland. The others either restrict residency to a contract tier or publish no region at all. | every plan, EU only | Enterprise only Geo Region Restriction, one region at a time | One region per account EU Central 1; multi-region is Enterprise | Not published residency controls advertised, no region named | Custom Device Lab only no region named | Not published | | Free trial, as published | 100 minutes across the whole platform, no card | Metered per product 30 Live, 60 Automate, 30 App Live, 100 App Automate | Not published trial offered, no length or allowance stated | Permanent free tier plus a trial | guided proof of concept of one to two weeks | Not published | Sources: browserstack.com/pricing, saucelabs.com/pricing, testmuai.com, testgrid.io/pricing and kobiton.com/pricing, all checked on 21 August 2026. LambdaTest rebranded to TestMu AI on 12 January 2026. Vendors change prices without notice, so check the source before you buy. TestingBot's own fleet is 40 physical devices and 6100+ browser and device combinations. At a glance ## What you get when you switch A single, transparent plan covers every BrowserStack module you currently pay extra for. - ### Flat, predictable pricing One plan, every feature, no surprise modules. The $50/mo automated plan gets you parallel testing, visual diffs, accessibility scans and codeless AI. - ### Single use VMs Every Selenium, Playwright or Cypress run gets a clean VM that is destroyed at the end. BrowserStack does not publish whether its VMs are single-use or recycled. - ### EU hosted, GDPR ready Test data, recordings and logs stay in Europe (EU and Switzerland). No transit to non-EU datacentres. Auditable by your DPO. - ### Real devices, no upcharge Run automated and manual tests on real iPhones and Android phones. Included in the plan, not a separate add-on. - ### Unlimited team seats Invite the whole engineering org. We do not bill per user. BrowserStack caps seats and upsells from there. - ### Faster, friendlier support European engineers on real support tickets, not bots. Median first response under one hour for paid plans. Free trial ## The BrowserStack free trial, and how it compares BrowserStack's free trial is metered in **minutes per product** rather than in days, and each product carries its own separate allowance. Their support pages publish no trial length in days, and no statement either way on whether a credit card is required. During the trial, browser and device access is limited: full access to their fleet comes only on purchase. ### What BrowserStack's free trial includes Quoted from BrowserStack's published FAQ and pricing page, checked 17 August 2026. - **Live (manual browser testing):** 30 minutes, 1 minute per device, plus 100 screenshots - **App Live (manual app testing):** 30 minutes, 1 minute per device - **Automate (browser automation):** 60 minutes - **App Automate (mobile automation):** 100 minutes - **Percy (visual testing):** 5,000 screenshots per month - **Device and browser access:** Limited during the trial; full access on purchase - **Credit card required:** Not published on their site - **What happens when it ends:** Not published on their site BrowserStack also runs permanently free $0 tiers for several products, including Percy, accessibility and test management, and a lifetime open source programme offering 5 users and 5 parallels. Those are separate from the trial above, and for Percy the free tier and the trial allowance are the same 5,000 screenshots. ### What TestingBot's free plan includes No credit card, and no separate allowance to keep track of per product. - **Testing minutes:** 100 minutes, usable across everything - **Manual browser testing:** Included - **Automated testing:** Included, 1 parallel - **Real iOS and Android devices:** Included - **Browser and device access:** The full grid, not a trial subset - **Credit card required:** No - **Expires after a fixed number of days:** No The practical difference is not the size of the allowance, it is the shape of it. BrowserStack meters each product separately and narrows the fleet you can reach; TestingBot gives one pool of minutes against the whole platform, so an evaluation that spans manual, automated and real-device testing does not need several trials running at once. [Start testing free](https://testingbot.com/users/sign_up) Compare ## TestingBot vs BrowserStack, feature by feature Everything BrowserStack charges extra for, side by side with what TestingBot includes. TestingBot vs BrowserStack, feature and pricing comparison.| Capability | Save 71% TestingBot | BrowserStack | | --- | --- | --- | | Automated testing, entry planAutomated testingEntry plan, 1 parallel | $50 / month Billed annually | $175 / month Automate Desktop & Mobile, per parallel, billed annually | | Real Devices for Automated Testing | | Billed separately | | Manual Live Testing | | | | Codeless AI Test Automation | | Billed separately | | Real Devices for Live Testing | | | | Single-use VMs (documented) TestingBot documents single-use VMs: every session gets a freshly provisioned machine that is destroyed afterwards. BrowserStack does not publish whether its VMs are single-use or recycled, so we make no claim either way. | | Not published | | Mobile Debugging | | | | Unlimited users on automation plans Both include unlimited users on their automation plans. BrowserStack prices Live, App Live and Test Management per user seat; TestingBot prices its Live plan per user tier. | | | | Linux as a test target BrowserStack's documented os capability accepts Windows and OS X only. Linux is supported for their Local client binary, which is a different thing. | | – | | Simulators & Emulators TestingBot offers both real devices and simulators/emulators. Simulators and emulators are useful for quick early-stage testing. | | – | | Visual Regression Testing | | Billed separately | | Accessibility Testing | | Billed separately | | EU hosting on every plan BrowserStack operates EU datacenters and offers Geo Region Restriction, but it is limited to Enterprise plans, applies to one region at a time, and excludes analytics, billing and test metadata. TestingBot hosts everything in the EU on every plan. | | Enterprise plans only | Migrate in an afternoon ## Switch from BrowserStack in four steps Your Selenium, Appium, Playwright and Cypress tests already speak W3C WebDriver. Point them at TestingBot and you're done. 1. 1 ### Create a free account Sign up at testingbot.com and copy your API key and secret from the dashboard. 2. 2 ### Point tests at TestingBot Swap the BrowserStack hub URL for hub.testingbot.com. Rename bstack:options to tb:options. That is the migration. 3. 3 ### Run a smoke test Launch a single Selenium or Appium test. Confirm browsers, devices and parallels are healthy, then move the rest of the suite. 4. 4 ### Claim 10% switch credit Send us a recent BrowserStack invoice via our [contact form](https://testingbot.com/contact/new). We apply the 10% loyalty discount to your plan. Coverage ## 6100+ real browsers and devices TestingBot provides thousands of real desktop browser and OS combinations, plus a curated fleet of physical iOS and Android devices. Just like BrowserStack, every popular target is available: Chrome, Firefox, Safari, Microsoft Edge, Opera and more, on Windows, macOS and Linux. Mobile coverage spans real iPhones, iPads and Android phones from iOS 12 and Android 7 onward, plus simulators and emulators for early-stage runs. [View supported browsers and devices](https://testingbot.com/support/web-automate/browsers) ![Real desktop browsers and mobile devices](https://testingbot.com/assets/alternatives/browsers-326041a6ac96d9681e3cf491f3cc00e81dc3e1fbcca91646da584cd787741286.jpg) Frameworks ## Bring your existing test suite First-class support for every popular test automation framework. No vendor lock-in. [![Selenium Testing](https://testingbot.com/assets/support/selenium-266ecc0599068fc6eca0bcd9174fb46053c95cebe1c66f7a50c480f3f1a52929.svg)](https://testingbot.com/features/automation/selenium) ### [Selenium](https://testingbot.com/features/automation/selenium) Industry-standard browser automation using WebDriver. Works with Java, Python, Ruby, JavaScript, C# and more. [![Appium Testing](https://testingbot.com/assets/support/appium-4b35a8bb56bae236bd84c539cee9b8bbb489a55ddf48e32d059c86685d230354.svg)](https://testingbot.com/features/automation/appium) ### [Appium](https://testingbot.com/features/automation/appium) Mobile automation on real iOS and Android devices, plus simulators and emulators when you need them. [![Cypress Testing](https://testingbot.com/assets/support/cypress-409c9fd3f79115ed7676aa1df4f6a217dc4402b3366a61984924aec7b4fefa97.svg)](https://testingbot.com/features/automation/cypress) ### [Cypress](https://testingbot.com/features/automation/cypress) Modern JavaScript end-to-end testing on Chrome, Firefox and Edge, with parallel execution. [![Puppeteer Testing](https://testingbot.com/assets/support/puppeteer-0a8cdd38af69dabd46638afbb3bffc8a372902949aa599663ec379f3e875263c.svg)](https://testingbot.com/features/automation/puppeteer) ### [Puppeteer](https://testingbot.com/features/automation/puppeteer) Headless Chrome and Edge automation from Google, ideal for fast scripted browser tasks. [![Playwright Testing](https://testingbot.com/assets/support/playwright-2dcb4e75b55ac0d1135cfa47be6137b13f46a2047ce78b96a34f51b0e2cc0d96.svg)](https://testingbot.com/features/automation/playwright) ### [Playwright](https://testingbot.com/features/automation/playwright) Microsoft's cross-browser automation framework with auto-waits, network interception and trace viewer. [![Espresso Testing](https://testingbot.com/assets/integrations/espresso-a8ff30da0c385f89f4e694bb29fc4779ad132e8e3f4a777a38e608e0492ff610.svg)](https://testingbot.com/features/automation/espresso) ### [Espresso](https://testingbot.com/features/automation/espresso) Google's instrumentation framework for native Android apps. Runs on the same real device fleet. [![XCUITest Testing](https://testingbot.com/assets/integrations/xcuitest-4a935f2368661d89d3559b3c5af3377236029478ae76bb43253f3c3bee712096.svg)](https://testingbot.com/features/automation/xcuitest) ### [XCUITest](https://testingbot.com/features/automation/xcuitest) Apple's native iOS UI testing framework. Run against real iPhones and iPads in our EU datacenter. [![Maestro Testing](https://testingbot.com/assets/integrations/maestro-9d782f30f13fb4efaf9928b704c7a787eb14cb9e5b20c650137bb65892334b8b.svg)](https://testingbot.com/features/automation/maestro) ### [Maestro](https://testingbot.com/features/automation/maestro) YAML-driven mobile UI testing. Quick to write, easy to maintain, and runs on real devices. Speed ## Parallel testing without the parallel pricing Cut total suite duration by running tests at the same time across single-use Windows, Linux and macOS VMs, all tuned for speed and stability. Each additional parallel slot on TestingBot costs less than half what BrowserStack charges, and every plan can scale without seat caps. [See pricing](https://testingbot.com/pricing) ![Parallel testing on TestingBot](https://testingbot.com/assets/features/selenium/parallel-0229d698e59a8a6b92fe50344a08634624f018331ecf8a849ef83cfb35e635a5.png) ![Codeless AI testing](https://testingbot.com/assets/home/ai-chat-24cc1ab8234690bad4a866888a2bd8e11f1301b7e21ad86be5d4d61be838707a.webp) AI testing, included ## Codeless AI Test Agent Describe the user journey in plain English. TestingBotAI writes the test, runs it on real browsers and devices, and keeps it working as your UI changes. No extra subscription, no separate workspace. Codeless AI is part of every TestingBot plan, where BrowserStack sells it as a separate product. [Start AI testing](https://testingbot.com/features/ai-testing) Why switch ## Why teams move from BrowserStack to TestingBot ### Transparent pricing BrowserStack splits cross-browser, visual, real-device, accessibility and AI into separate paid modules. TestingBot's plan covers all of them, with no surprise upsells. ### Support that answers Real engineers, not bots. Median first response under one hour on paid plans, and direct lines for enterprise accounts. ### Cleaner UX Fewer clicks to start a session, fewer dashboards to learn, and a single inbox for desktop, mobile and AI test runs. ### Emulators and simulators BrowserStack sells real devices only: no emulator or simulator session appears in its App Live or App Automate plans, and its own emulator pages argue for real devices instead. TestingBot pairs real devices with simulators and emulators so quick local-style runs are possible too. ### Single-use security Every TestingBot VM is destroyed at the end of a session, which we document. BrowserStack does not publish whether its VMs are single-use or recycled. ### Hosted in Europe TestingBot is built and operated entirely in the EU. Your test data stays inside European borders. BrowserStack does operate EU datacentres, but EU data residency is a Geo Region Restriction feature limited to Enterprise plans, applies to one region at a time, and excludes analytics, billing and test metadata. > "Users often find that BrowserStack can overwhelm them with complex UX and lack of support. Others cite frustrations with BrowserStack's high costs once you use multiple features, making it tricky to get full coverage without breaking the budget." _G2 Reviewers_ FAQ ## Frequently asked questions Is TestingBot really a cheaper BrowserStack alternative? Yes. TestingBot's entry plan for automated testing starts at $50 per month for one parallel, billed annually, against the comparable BrowserStack Automate Desktop and Mobile plan at $175 per parallel, a 71% saving. Visual testing, accessibility scans, codeless AI and real devices are all included in that subscription. BrowserStack sells Percy and Accessibility Testing as separate, sales-led products, and puts real devices behind a higher Automate tier. How long does it take to migrate from BrowserStack to TestingBot? Most teams migrate in a single afternoon. The Selenium, Appium, Playwright and Cypress endpoints follow the same W3C contract as BrowserStack, so the change is typically a hub URL and a capability prefix. Can TestingBot replace BrowserStack, Percy and LambdaTest at once? Yes. TestingBot is a unified platform for manual, automated, visual, accessibility and AI testing on real desktop browsers and physical mobile devices. One subscription replaces all three. Is TestingBot hosted in Europe? Is it GDPR compliant? Yes. TestingBot is operated entirely from Europe. Test data, recordings and logs stay in Europe (EU and Switzerland, covered by a European Commission adequacy decision), which makes the platform a GDPR friendly alternative. BrowserStack does operate EU datacentres, but EU data residency is a Geo Region Restriction feature limited to Enterprise plans, applies to one region at a time, and excludes analytics, billing and test metadata. Does TestingBot offer a free plan? Yes. New accounts include 100 minutes of testing across desktop browsers and real mobile devices, with one parallel test. No credit card is required to start, and the minutes are not tied to a fixed number of days. Does BrowserStack have a free trial, and how long is it? BrowserStack offers a free trial, but it is metered in minutes per product rather than in days, and they publish no trial length in days. Their FAQ lists 30 minutes for Live, 60 minutes for Automate, 30 minutes for App Live, 100 minutes for App Automate and 5,000 Percy screenshots per month, each as a separate allowance. Checked on browserstack.com on 17 August 2026. Is BrowserStack free, or does it only offer a free trial? Both. Alongside the time-limited trial, BrowserStack publishes permanently free $0 tiers for several products, including Percy, accessibility testing and test management, plus a lifetime open source programme with 5 users and 5 parallels. For Percy the free tier and the trial allowance are the same 5,000 screenshots per month. Does the BrowserStack free trial require a credit card? BrowserStack does not state either way on their own site, so we make no claim about it. TestingBot does not require a credit card: you get 100 minutes of testing on sign-up, and a card is only needed if you later start the optional 7-day extended trial of a paid plan. What are the limits of the BrowserStack free trial? Beyond the per-product minutes, BrowserStack states that free trials of Live have limited browser and device access, and that full access to their fleet comes on purchase. What happens once a trial is exhausted is not published on their site. TestingBot's free minutes run against the full grid rather than a trial subset. What test frameworks does TestingBot support? TestingBot supports Selenium, Appium, Playwright, Cypress, Puppeteer, WebdriverIO, Espresso, XCUITest, Maestro and Selenide. The same endpoint serves desktop browsers and mobile devices. Does TestingBot provide real iOS and Android devices? Yes. A fleet of real iPhones, iPads and Android phones is available for manual and automated testing. Real devices for automated testing are included in the standard plan, not billed separately as on BrowserStack. Why would I choose TestingBot over BrowserStack? Flat pricing that does not punish scale, single-use VMs that never reuse machines across customers, EU hosted data with full GDPR compliance, and faster human support. ## Ready to ditch the BrowserStack invoice? Real browsers, real devices, one plan, EU hosted. Free plan, no card required. [Start free trial](https://testingbot.com/users/sign_up) --- URL: https://testingbot.com/lambdatest-alternative TestingBot vs LambdaTest # The affordable LambdaTest alternative LambdaTest rebranded to **TestMu AI** in January 2026. The platform, the pricing and the trade-offs are unchanged. This page covers both names. Real desktop browsers, physical iPhones and Android phones, codeless AI, visual and accessibility testing. One transparent plan. GDPR hosted in the EU. Already on LambdaTest or TestMu AI? Send us a recent invoice and we'll knock **10% off** your TestingBot subscription for the switch. [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) - **Tests executed:** 1+ bn - **Active users:** 180k - **Browsers & devices:** 6100+ Switching LambdaTest / TestMu TestingBot Save 75% $78/mo saved 1 day to switch GDPR · EU hosting Trusted by some of the world's most innovative companies The shortlist ## Six cross-browser and device testing platforms, compared Prices and capabilities below are taken from each vendor's own pricing pages, checked on 21 August 2026. Where a vendor does not publish something we say so rather than guessing. TestingBot, BrowserStack, Sauce Labs, TestMu AI, TestGrid and Kobiton compared on pricing, metering, device coverage, hosting and free trials.| Capability | TestingBot | [BrowserStack](https://testingbot.com/browserstack-alternative) | [Sauce Labs](https://testingbot.com/saucelabs-alternative) | TestMu AI | TestGrid | Kobiton | | --- | --- | --- | --- | --- | --- | --- | | Cheapest plan that includes real device automation Annual billing. TestingBot publishes in euro, the others in US dollars, so these are list prices and not a currency conversion. | EUR 50 Automated, 1 parallel | $175 Automate Desktop & Mobile, per parallel | $199 Real Device Cloud | $199 Real Device Plus Automation, per parallel | $199 Starter, per seat, 4 devices or browsers | From $83 Startup, 500 minutes per month | | How usage is metered | Parallel sessions | Parallel sessions | Parallel sessions | Parallel sessions | Seats plus AI tokens | Minutes per month | | Emulators and simulators alongside real devices BrowserStack publishes no emulator or simulator option in App Live or App Automate. TestGrid and Kobiton publish real devices and real browsers only, and name no emulator or simulator product. | | | 1700+ published | | Not published | Not published | | EU hosting or data residency TestingBot runs every plan from the EU and Switzerland. The others either restrict residency to a contract tier or publish no region at all. | every plan, EU only | Enterprise only Geo Region Restriction, one region at a time | One region per account EU Central 1; multi-region is Enterprise | Not published residency controls advertised, no region named | Custom Device Lab only no region named | Not published | | Free trial, as published | 100 minutes across the whole platform, no card | Metered per product 30 Live, 60 Automate, 30 App Live, 100 App Automate | Not published trial offered, no length or allowance stated | Permanent free tier plus a trial | guided proof of concept of one to two weeks | Not published | Sources: browserstack.com/pricing, saucelabs.com/pricing, testmuai.com, testgrid.io/pricing and kobiton.com/pricing, all checked on 21 August 2026. LambdaTest rebranded to TestMu AI on 12 January 2026. Vendors change prices without notice, so check the source before you buy. TestingBot's own fleet is 40 physical devices and 6100+ browser and device combinations. At a glance ## What you get when you switch A single, transparent plan covers every LambdaTest module you currently pay extra for. - ### Flat, predictable pricing One plan covers parallel testing, codeless AI, visual diffs, accessibility scans and real devices from $50/mo. No surprise add-ons. - ### Single use VMs Every Selenium, Playwright or Cypress run gets a clean VM that is destroyed at the end. LambdaTest does not publish whether its VMs are single-use or recycled. - ### EU hosted, GDPR ready Test data, recordings and logs stay in Europe (EU and Switzerland). No transit to non-EU datacentres. Auditable by your DPO. - ### Real devices, no upcharge Run automated and manual tests on real iPhones and Android phones. Included in the plan, not a separate add-on like on LambdaTest. - ### Codeless AI, included Describe the user flow in plain English. TestingBotAI writes the test, runs it and keeps it green. LambdaTest charges extra for KaneAI. - ### Faster, friendlier support European engineers on real tickets, with median first response under one hour on paid plans. Detailed documentation that matches the product. Compare ## TestingBot vs LambdaTest, feature by feature Everything LambdaTest charges extra for, side by side with what TestingBot includes. TestingBot vs LambdaTest, feature and pricing comparison.| Capability | Save 75% TestingBot | LambdaTest | | --- | --- | --- | | Automated testing, entry planAutomated testingEntry plan, 1 parallel | $50 / month Billed annually | $199 / month Real Device Plus Automation Cloud Billed annually | | Real Devices for Automated Testing | | Higher-priced plan | | Real Devices for Live Testing | | | | Codeless AI Test Automation | | Billed separately | | Native App Automation | | Higher-priced plan | | Manual Live Testing | | | | Single-use VMs (documented) TestingBot documents single-use VMs: every session gets a freshly provisioned machine that is destroyed afterwards. LambdaTest does not publish whether its VMs are single-use or recycled, so we make no claim either way. | | Not published | | Mobile Debugging | | | | Linux virtual machines TestMu AI sells a Web Automation on Desktop (Linux Only) plan at $29/month and a HyperExecute Cloud (Linux) plan, so Linux is available on both sides. They do not publish which distributions. | | | | Visual Regression Testing | | Billed separately | | Accessibility Testing | | Separate product | | EU hosting, named and on every plan TestingBot names its hosting: Belgium, Germany, Finland and Switzerland. TestMu AI names AWS and Azure as hosts but publishes no data centre region; residency is described as an enterprise control, and their privacy policy relies on EU Standard Contractual Clauses for EU-to-US transfers. | | Not published | Migrate in an afternoon ## Switch from LambdaTest in four steps Your Selenium, Appium, Playwright and Cypress tests already speak W3C WebDriver. Point them at TestingBot and you're done. 1. 1 ### Create a free account Sign up at testingbot.com and copy your API key and secret from the dashboard. 2. 2 ### Point tests at TestingBot Swap the LambdaTest hub URL for hub.testingbot.com. Rename LT:Options to tb:options. That is the migration. 3. 3 ### Run a smoke test Launch a single Selenium or Appium test. Confirm browsers, devices and parallels are healthy, then move the rest of the suite. 4. 4 ### Claim 10% switch credit Send us a recent LambdaTest invoice via our [contact form](https://testingbot.com/contact/new). We apply the 10% loyalty discount to your plan. Coverage ## 6100+ real browsers and devices TestingBot provides thousands of real desktop browser and OS combinations, plus a curated fleet of physical iOS and Android devices. Just like LambdaTest, every popular target is available: Chrome, Firefox, Safari, Microsoft Edge, Opera and more, on Windows, macOS and Linux. Mobile coverage spans real iPhones, iPads and Android phones from iOS 12 and Android 7 onward, plus simulators and emulators for early-stage runs. [View supported browsers and devices](https://testingbot.com/support/web-automate/browsers) ![Real desktop browsers and mobile devices](https://testingbot.com/assets/alternatives/browsers-326041a6ac96d9681e3cf491f3cc00e81dc3e1fbcca91646da584cd787741286.jpg) Frameworks ## Bring your existing test suite First-class support for every popular test automation framework. No vendor lock-in. [![Selenium Testing](https://testingbot.com/assets/support/selenium-266ecc0599068fc6eca0bcd9174fb46053c95cebe1c66f7a50c480f3f1a52929.svg)](https://testingbot.com/features/automation/selenium) ### [Selenium](https://testingbot.com/features/automation/selenium) Industry-standard browser automation using WebDriver. Works with Java, Python, Ruby, JavaScript, C# and more. [![Appium Testing](https://testingbot.com/assets/support/appium-4b35a8bb56bae236bd84c539cee9b8bbb489a55ddf48e32d059c86685d230354.svg)](https://testingbot.com/features/automation/appium) ### [Appium](https://testingbot.com/features/automation/appium) Mobile automation on real iOS and Android devices, plus simulators and emulators when you need them. [![Cypress Testing](https://testingbot.com/assets/support/cypress-409c9fd3f79115ed7676aa1df4f6a217dc4402b3366a61984924aec7b4fefa97.svg)](https://testingbot.com/features/automation/cypress) ### [Cypress](https://testingbot.com/features/automation/cypress) Modern JavaScript end-to-end testing on Chrome, Firefox and Edge, with parallel execution. [![Puppeteer Testing](https://testingbot.com/assets/support/puppeteer-0a8cdd38af69dabd46638afbb3bffc8a372902949aa599663ec379f3e875263c.svg)](https://testingbot.com/features/automation/puppeteer) ### [Puppeteer](https://testingbot.com/features/automation/puppeteer) Headless Chrome and Edge automation from Google, ideal for fast scripted browser tasks. [![Playwright Testing](https://testingbot.com/assets/support/playwright-2dcb4e75b55ac0d1135cfa47be6137b13f46a2047ce78b96a34f51b0e2cc0d96.svg)](https://testingbot.com/features/automation/playwright) ### [Playwright](https://testingbot.com/features/automation/playwright) Microsoft's cross-browser automation framework with auto-waits, network interception and trace viewer. [![Espresso Testing](https://testingbot.com/assets/integrations/espresso-a8ff30da0c385f89f4e694bb29fc4779ad132e8e3f4a777a38e608e0492ff610.svg)](https://testingbot.com/features/automation/espresso) ### [Espresso](https://testingbot.com/features/automation/espresso) Google's instrumentation framework for native Android apps. Runs on the same real device fleet. [![XCUITest Testing](https://testingbot.com/assets/integrations/xcuitest-4a935f2368661d89d3559b3c5af3377236029478ae76bb43253f3c3bee712096.svg)](https://testingbot.com/features/automation/xcuitest) ### [XCUITest](https://testingbot.com/features/automation/xcuitest) Apple's native iOS UI testing framework. Run against real iPhones and iPads in our EU datacenter. [![Maestro Testing](https://testingbot.com/assets/integrations/maestro-9d782f30f13fb4efaf9928b704c7a787eb14cb9e5b20c650137bb65892334b8b.svg)](https://testingbot.com/features/automation/maestro) ### [Maestro](https://testingbot.com/features/automation/maestro) YAML-driven mobile UI testing. Quick to write, easy to maintain, and runs on real devices. Speed ## Parallel testing without the parallel pricing Cut total suite duration by running tests at the same time across single-use Windows, Linux and macOS VMs, all tuned for speed and stability. Each additional parallel slot on TestingBot costs noticeably less than LambdaTest, and every plan can scale without seat caps. [See pricing](https://testingbot.com/pricing) ![Parallel testing on TestingBot](https://testingbot.com/assets/features/selenium/parallel-0229d698e59a8a6b92fe50344a08634624f018331ecf8a849ef83cfb35e635a5.png) ![Codeless AI testing](https://testingbot.com/assets/home/ai-chat-24cc1ab8234690bad4a866888a2bd8e11f1301b7e21ad86be5d4d61be838707a.webp) AI testing, included ## Codeless AI Test Agent Describe the user journey in plain English. TestingBotAI writes the test, runs it on real browsers and devices, and keeps it working as your UI changes. No extra subscription, no separate workspace. Codeless AI is part of every TestingBot plan, where LambdaTest (TestMu AI) sells KaneAI as a separately priced add-on with its own seat and consumption limits. [Start AI testing](https://testingbot.com/features/ai-testing) Why switch ## Why teams move from LambdaTest to TestingBot ### Transparent pricing LambdaTest (TestMu AI) splits cross-browser, visual, real-device, accessibility and KaneAI into separate paid modules. TestingBot's plan covers all of them, with no surprise upsells. ### Support that answers European engineers, median first response under one hour on paid plans. LambdaTest users describe inconsistent support coverage and slow ticket resolution. ### Cleaner UX Fewer clicks to start a session, fewer dashboards to learn, and a single inbox for desktop, mobile and AI test runs. ### Documentation that helps Clear getting-started guides, working code samples and integrations. LambdaTest's docs are often incomplete or out of sync with the product. ### Single-use security Every TestingBot VM is destroyed at the end of a session, which we document. LambdaTest does not publish whether its VMs are single-use or recycled. ### Hosted in Europe TestingBot is built and operated entirely in the EU. Your test data stays inside European borders. TestMu AI hosts on AWS and Azure without disclosing regions, and runs a product development centre in India. > "Users often find that LambdaTest is slow to use, with slow startup times for browsers. Others cite frustrations with LambdaTest's pricing structure when using multiple features, making it difficult to get full coverage without unexpected expenses." _G2 Reviewers_ FAQ ## Frequently asked questions Is LambdaTest the same company as TestMu AI? Yes. LambdaTest rebranded to TestMu AI on 12 January 2026. The company, platform, pricing, KaneAI product and infrastructure are unchanged. Existing accounts, API keys and configuration files continue to work under the new name. The TestingBot vs TestMu AI comparison is identical to the TestingBot vs LambdaTest comparison on this page. Is TestingBot really a cheaper LambdaTest alternative? TestingBot's Automated plan is $50 per month with real devices, codeless AI, visual testing and accessibility scans included. The comparable TestMu AI plan, Real Device Plus Automation Cloud, is $199 per month billed annually, with SmartUI and the Accessibility Testing Agent sold as separate products. Their cheaper automation tiers start at $29 per month but exclude real devices. Verified on testmuai.com on 17 August 2026. How long does it take to migrate from LambdaTest to TestingBot? Most teams migrate in a single afternoon. The Selenium, Appium, Playwright and Cypress endpoints follow the same W3C contract as LambdaTest, so the change is typically a hub URL and a capability prefix. Can TestingBot replace LambdaTest, BrowserStack and Sauce Labs at once? Yes. TestingBot is a unified platform for manual, automated, visual, accessibility and AI testing on real desktop browsers and physical mobile devices. One subscription replaces all three. Is TestingBot hosted in Europe? Is it GDPR compliant? Yes. TestingBot is operated entirely from Europe. Test data, recordings and logs stay in Europe (EU and Switzerland, covered by a European Commission adequacy decision), which makes the platform a GDPR friendly alternative. TestMu AI names AWS and Azure as its hosts but publishes no data centre region, and its privacy policy relies on EU Standard Contractual Clauses for EU-to-US transfers. Does TestingBot offer a free plan? Yes. New accounts include free minutes to evaluate the platform across desktop browsers and real mobile devices. No credit card is required to start. What test frameworks does TestingBot support? TestingBot supports Selenium, Appium, Playwright, Cypress, Puppeteer, WebdriverIO, Espresso, XCUITest, Maestro and Selenide. The same endpoint serves desktop browsers and mobile devices. Does TestingBot provide real iOS and Android devices? Yes. A fleet of real iPhones, iPads and Android phones is available for manual and automated testing. Real devices are included in the standard plan, not billed separately as on LambdaTest. Why would I choose TestingBot over LambdaTest? Flat pricing that does not punish scale, single-use VMs that never reuse machines across customers, EU hosted data with full GDPR compliance, codeless AI included, and faster human support. ## Ready to leave the LambdaTest invoice behind? Real browsers, real devices, one plan, EU hosted. Free plan, no card required. [Start free trial](https://testingbot.com/users/sign_up) --- URL: https://testingbot.com/saucelabs-alternative TestingBot vs Sauce Labs # The affordable Sauce Labs alternative **TestingBot is the most affordable Sauce Labs alternative** , starting at **$90 per month** with EU hosting, real mobile devices and codeless AI included. Run Selenium, Appium, Playwright, Cypress, Espresso, XCUITest and Maestro tests on 6100+ real browsers and physical iOS and Android devices on a single GDPR-hosted plan. Already on Sauce Labs? Send us a recent invoice and we'll knock **10% off** your TestingBot subscription for the switch. [Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo) - **Tests executed:** 1+ bn - **Active users:** 180k - **Browsers & devices:** 6100+ Switching Sauce Labs TestingBot Save 40% $59/mo saved 1 day to switch GDPR · EU hosting Trusted by some of the world's most innovative companies The shortlist ## Six cross-browser and device testing platforms, compared Prices and capabilities below are taken from each vendor's own pricing pages, checked on 21 August 2026. Where a vendor does not publish something we say so rather than guessing. TestingBot, BrowserStack, Sauce Labs, TestMu AI, TestGrid and Kobiton compared on pricing, metering, device coverage, hosting and free trials.| Capability | TestingBot | [BrowserStack](https://testingbot.com/browserstack-alternative) | Sauce Labs | [TestMu AI](https://testingbot.com/lambdatest-alternative) | TestGrid | Kobiton | | --- | --- | --- | --- | --- | --- | --- | | Cheapest plan that includes real device automation Annual billing. TestingBot publishes in euro, the others in US dollars, so these are list prices and not a currency conversion. | EUR 50 Automated, 1 parallel | $175 Automate Desktop & Mobile, per parallel | $199 Real Device Cloud | $199 Real Device Plus Automation, per parallel | $199 Starter, per seat, 4 devices or browsers | From $83 Startup, 500 minutes per month | | How usage is metered | Parallel sessions | Parallel sessions | Parallel sessions | Parallel sessions | Seats plus AI tokens | Minutes per month | | Emulators and simulators alongside real devices BrowserStack publishes no emulator or simulator option in App Live or App Automate. TestGrid and Kobiton publish real devices and real browsers only, and name no emulator or simulator product. | | | 1700+ published | | Not published | Not published | | EU hosting or data residency TestingBot runs every plan from the EU and Switzerland. The others either restrict residency to a contract tier or publish no region at all. | every plan, EU only | Enterprise only Geo Region Restriction, one region at a time | One region per account EU Central 1; multi-region is Enterprise | Not published residency controls advertised, no region named | Custom Device Lab only no region named | Not published | | Free trial, as published | 100 minutes across the whole platform, no card | Metered per product 30 Live, 60 Automate, 30 App Live, 100 App Automate | Not published trial offered, no length or allowance stated | Permanent free tier plus a trial | guided proof of concept of one to two weeks | Not published | Sources: browserstack.com/pricing, saucelabs.com/pricing, testmuai.com, testgrid.io/pricing and kobiton.com/pricing, all checked on 21 August 2026. LambdaTest rebranded to TestMu AI on 12 January 2026. Vendors change prices without notice, so check the source before you buy. TestingBot's own fleet is 40 physical devices and 6100+ browser and device combinations. At a glance ## What you get when you switch A single, transparent plan covers every Sauce Labs add-on you currently pay extra for. - ### Flat, predictable pricing One plan covers parallel testing, codeless AI, visual diffs, accessibility scans and real devices. $90/mo, no surprise add-ons. - ### Real human support European engineers on real support tickets. Median first response under one hour on paid plans, not days in a queue. - ### EU hosted, GDPR ready Test data, recordings and logs stay in Europe (EU and Switzerland). No transit to US datacentres. Auditable by your DPO. - ### Real devices, no upcharge Run automated and manual tests on real iPhones and Android phones. Included in the plan, not a separate add-on like on Sauce Labs. - ### Codeless AI, included Describe the user flow in plain English. TestingBotAI writes the test, runs it and keeps it green, on every plan. Sauce Labs ships a comparable tool in Sauce AI for Test Authoring, but their docs gate it behind a paid Enterprise add-on. - ### Shipping faster TestingBot shipped Apple Silicon macOS, real iOS 26 and AI test creation months ahead of Sauce Labs. The roadmap moves. Compare ## TestingBot vs Sauce Labs, feature by feature Everything Sauce Labs charges extra for or gates behind an Enterprise contract, side by side with what TestingBot includes. TestingBot vs Sauce Labs, feature and pricing comparison.| Capability | Save 40% TestingBot | Sauce Labs | | --- | --- | --- | | Automated testing, comparable planAutomated testing1 parallel | $90 / month Billed annually | $149 / month Virtual Device Cloud, annual Billed annually | | Real Devices for Automated Testing | | Billed separately | | Codeless AI Test Automation Sauce AI for Test Authoring builds tests from natural-language prompts, but their docs state it is a paid add-on that requires an Enterprise account and available device concurrency. TestingBot includes codeless AI on every plan. | | Enterprise add-on | | Visual Regression Testing | | | | Accessibility Testing Sauce Labs offers accessibility testing through an integration with Deque axe. No price or plan entitlement is published and the only call to action is to contact sales. TestingBot includes accessibility scans in the plan price. | | Contact sales | | Single-Use VMs | | | | Mobile Debugging | | | | Unlimited users on automation plans | | | | EU hosting on every plan Sauce Labs documents an EU Central 1 region alongside two US regions. Self-service and trial accounts choose a single data centre at sign-up and cannot use EU and US together; only Enterprise contracts cover more than one. TestingBot hosts everything in the EU on every plan. | | One region only | | Virtual and real devices in one plan Sauce Labs splits coverage across plans: Virtual Device Cloud at $149/month covers desktop browsers, emulators and simulators, while real devices need Real Device Cloud at $199/month. TestingBot includes both in one plan. | | Two plans | Track record ## Shipping ahead, not behind Concrete platform milestones where TestingBot landed months before Sauce Labs, or where Sauce Labs has nothing equivalent to ship at all. 10 months ### Ahead on Apple Silicon macOS TestingBot offers Apple Silicon (M-series) macOS virtual machines, so Safari and iOS Catalyst suites run on the same architecture your users have. Launch day ### iOS 26 the week Apple shipped it Real iPhones and iPads running iOS 26 were available on TestingBot the same week Apple released the OS. No queue, no waiting, no missed regressions on the newest Safari. 2025 ### Codeless AI testing, generally available TestingBot's codeless AI test agent has been live for paying customers since 2025. Sauce AI for Test Authoring is gated behind a paid Enterprise add-on. Building the platform since **2011**. Same team, same focus on cross-browser and mobile testing. Migrate in an afternoon ## Switch from Sauce Labs in four steps Your Selenium, Appium, Playwright and Cypress tests already speak W3C WebDriver. Point them at TestingBot and you're done. 1. 1 ### Create a free account Sign up at testingbot.com and copy your API key and secret from the dashboard. 2. 2 ### Point tests at TestingBot Swap the Sauce Labs hub URL for hub.testingbot.com. Rename sauce:options to tb:options. That is the migration. 3. 3 ### Run a smoke test Launch a single Selenium or Appium test. Confirm browsers, devices and parallels are healthy, then move the rest of the suite. 4. 4 ### Claim 10% switch credit Send us a recent Sauce Labs invoice via our [contact form](https://testingbot.com/contact/new). We apply the 10% loyalty discount to your plan. Coverage ## 6100+ real browsers and devices TestingBot provides thousands of real desktop browser and OS combinations, plus a curated fleet of physical iOS and Android devices. Just like Sauce Labs, every popular target is available: Chrome, Firefox, Safari, Microsoft Edge, Opera and more on Windows, macOS and Linux, including Apple Silicon machines. Mobile coverage spans real iPhones, iPads and Android phones from iOS 12 and Android 7 onward, plus simulators and emulators for early-stage runs. [View supported browsers and devices](https://testingbot.com/support/web-automate/browsers) ![Real desktop browsers and mobile devices](https://testingbot.com/assets/alternatives/browsers-326041a6ac96d9681e3cf491f3cc00e81dc3e1fbcca91646da584cd787741286.jpg) Frameworks ## Bring your existing test suite First-class support for every popular test automation framework. No vendor lock-in. [![Selenium Testing](https://testingbot.com/assets/support/selenium-266ecc0599068fc6eca0bcd9174fb46053c95cebe1c66f7a50c480f3f1a52929.svg)](https://testingbot.com/features/automation/selenium) ### [Selenium](https://testingbot.com/features/automation/selenium) Industry-standard browser automation using WebDriver. Works with Java, Python, Ruby, JavaScript, C# and more. [![Appium Testing](https://testingbot.com/assets/support/appium-4b35a8bb56bae236bd84c539cee9b8bbb489a55ddf48e32d059c86685d230354.svg)](https://testingbot.com/features/automation/appium) ### [Appium](https://testingbot.com/features/automation/appium) Mobile automation on real iOS and Android devices, plus simulators and emulators when you need them. [![Cypress Testing](https://testingbot.com/assets/support/cypress-409c9fd3f79115ed7676aa1df4f6a217dc4402b3366a61984924aec7b4fefa97.svg)](https://testingbot.com/features/automation/cypress) ### [Cypress](https://testingbot.com/features/automation/cypress) Modern JavaScript end-to-end testing on Chrome, Firefox and Edge, with parallel execution. [![Puppeteer Testing](https://testingbot.com/assets/support/puppeteer-0a8cdd38af69dabd46638afbb3bffc8a372902949aa599663ec379f3e875263c.svg)](https://testingbot.com/features/automation/puppeteer) ### [Puppeteer](https://testingbot.com/features/automation/puppeteer) Headless Chrome and Edge automation from Google, ideal for fast scripted browser tasks. [![Playwright Testing](https://testingbot.com/assets/support/playwright-2dcb4e75b55ac0d1135cfa47be6137b13f46a2047ce78b96a34f51b0e2cc0d96.svg)](https://testingbot.com/features/automation/playwright) ### [Playwright](https://testingbot.com/features/automation/playwright) Microsoft's cross-browser automation framework with auto-waits, network interception and trace viewer. [![Espresso Testing](https://testingbot.com/assets/integrations/espresso-a8ff30da0c385f89f4e694bb29fc4779ad132e8e3f4a777a38e608e0492ff610.svg)](https://testingbot.com/features/automation/espresso) ### [Espresso](https://testingbot.com/features/automation/espresso) Google's instrumentation framework for native Android apps. Runs on the same real device fleet. [![XCUITest Testing](https://testingbot.com/assets/integrations/xcuitest-4a935f2368661d89d3559b3c5af3377236029478ae76bb43253f3c3bee712096.svg)](https://testingbot.com/features/automation/xcuitest) ### [XCUITest](https://testingbot.com/features/automation/xcuitest) Apple's native iOS UI testing framework. Run against real iPhones and iPads in our EU datacenter. [![Maestro Testing](https://testingbot.com/assets/integrations/maestro-9d782f30f13fb4efaf9928b704c7a787eb14cb9e5b20c650137bb65892334b8b.svg)](https://testingbot.com/features/automation/maestro) ### [Maestro](https://testingbot.com/features/automation/maestro) YAML-driven mobile UI testing. Quick to write, easy to maintain, and runs on real devices. Speed ## Parallel testing without the parallel pricing Cut total suite duration by running tests at the same time across single-use Windows, Linux and macOS VMs, all tuned for speed and stability. Each additional parallel slot on TestingBot costs noticeably less than Sauce Labs, and every plan can scale without seat caps. [See pricing](https://testingbot.com/pricing) ![Parallel testing on TestingBot](https://testingbot.com/assets/features/selenium/parallel-0229d698e59a8a6b92fe50344a08634624f018331ecf8a849ef83cfb35e635a5.png) ![Codeless AI testing](https://testingbot.com/assets/home/ai-chat-24cc1ab8234690bad4a866888a2bd8e11f1301b7e21ad86be5d4d61be838707a.webp) Enterprise-only at Sauce Labs ## Codeless AI Test Agent Describe the user journey in plain English. TestingBotAI writes the test, runs it on real browsers and devices, and keeps it working as your UI changes. Included in every TestingBot plan. Sauce Labs has a comparable tool in Sauce AI for Test Authoring, but their documentation states it is a paid add-on that requires an Enterprise account. [Start AI testing](https://testingbot.com/features/ai-testing) Why switch ## Why teams move from Sauce Labs to TestingBot ### Transparent pricing Real devices, codeless AI, accessibility and visual testing are all in one plan from $90/mo. Sauce Labs invoices grow with every add-on. ### Support that answers You reach the European engineers who build the platform, not a first-line queue. No account manager required to get a bug looked at. ### New features ship faster Apple Silicon macOS, real iOS 26 devices and codeless AI test creation all reached TestingBot users as standard features rather than paid add-ons. ### Real devices included Run automated and manual Appium, XCUITest, Espresso and Maestro tests on real iPhones and Android phones without a separate device plan. ### Documentation that helps Clear getting-started guides, working code samples and integrations for CI/CD. Migration from Sauce Labs is usually one afternoon. ### Hosted in Europe TestingBot is built and operated entirely in the EU. Your test data stays inside European borders. Sauce Labs runs from the United States. > "Flat pricing and EU-hosted devices were the two things we could not get elsewhere. Support answers in the same working day, from someone who has read the test." _G2 Reviewers_ FAQ ## Frequently asked questions Is TestingBot really a cheaper Sauce Labs alternative? TestingBot is $90 per month against the $149 Sauce Labs charges for Virtual Device Cloud, roughly 40% cheaper. Codeless AI, visual diffs, accessibility scans, real devices and parallel slots are all included in that price. At Sauce Labs, real devices need the separate $199 Real Device Cloud plan, codeless AI is a paid Enterprise add-on and accessibility is contact-sales. Their $39 Live Testing plan is cheaper but covers manual testing only. Verified on saucelabs.com on 17 August 2026. How long does it take to migrate from Sauce Labs to TestingBot? Most teams migrate in a single afternoon. The Selenium, Appium, Playwright and Cypress endpoints follow the same W3C contract as Sauce Labs, so the change is typically a hub URL and a capability prefix. Can TestingBot replace Sauce Labs, BrowserStack and LambdaTest at once? Yes. TestingBot is a unified platform for manual, automated, visual, accessibility and AI testing on real desktop browsers and physical mobile devices. One subscription replaces all three. Is TestingBot hosted in Europe? Is it GDPR compliant? Yes. TestingBot is operated entirely from Europe. Test data, recordings and logs stay in Europe (EU and Switzerland, covered by a European Commission adequacy decision), which makes the platform a GDPR friendly alternative. Sauce Labs documents an EU Central 1 region, but self-service and trial accounts choose a single data centre at sign-up and only Enterprise contracts cover more than one. Does TestingBot offer a free plan? Yes. New accounts include free minutes to evaluate the platform across desktop browsers and real mobile devices. No credit card is required to start. What test frameworks does TestingBot support? TestingBot supports Selenium, Appium, Playwright, Cypress, Puppeteer, WebdriverIO, Espresso, XCUITest, Maestro and Selenide. The same endpoint serves desktop browsers and mobile devices. Does TestingBot provide real iOS and Android devices? Yes. A fleet of real iPhones, iPads and Android phones is available for manual and automated testing. Real devices are included in the standard plan, not billed separately as on Sauce Labs. Why would I choose TestingBot over Sauce Labs? Lower flat pricing, real human support with response in under an hour on paid plans, codeless AI test creation on every plan, where Sauce Labs gates its equivalent behind a paid Enterprise add-on, and EU hosted data with full GDPR compliance. ## Ready to leave the Sauce Labs queue behind? Real browsers, real devices, real support. One plan, EU hosted. Free plan, no card required. [Start free trial](https://testingbot.com/users/sign_up) --- URL: https://testingbot.com/demo Talk to a TestingBot expert # Get a personalized TestingBot demo Book a 15 minute call with our team. We will tailor the demo to your tech stack, answer pricing questions and help you decide if TestingBot is the right fit. 1+bn Tests Executed 180k Active Users 6100+ Browsers & Devices ## What you will see in the demo - A live test running on a real iOS or Android device, plus the desktop browser grid. - Codeless automation, visual regression testing and manual live testing in one platform. - How TestingBot plugs into your CI (GitHub Actions, Jenkins, GitLab, CircleCI) and existing frameworks. - EU hosted infrastructure, security posture and pricing tailored to your team size. Prefer email? [Send us your questions](https://testingbot.com/contact/new) and we will get back to you within one business day. Calendar not loading? [Open the booking page directly](https://calendly.com/testingbot/15min). Trusted by some of the world's most innovative companies EU hosted Data stays in Europe GDPR & CCPA Privacy compliant SAML SSO Single Sign-On ready 99.9% uptime Reliable at scale ## Frequently asked questions How long is the demo? Most demos take 15 to 30 minutes. We adjust the agenda to whatever you want to focus on, whether that is real device testing, codeless automation, CI integration or pricing. Who will I speak with? You will talk directly with a TestingBot engineer who can answer technical questions on the spot. No SDR handoffs, no scripted pitches. Do I need an account to book a demo? No. You can pick a time on the calendar without signing up. If you already have a free trial running we can use that during the call, otherwise we will spin up a sandbox for you. Can we run a custom proof of concept? Yes. For teams evaluating TestingBot at scale we offer extended trials with engineering support, custom device allocations and migration help from BrowserStack, Sauce Labs or LambdaTest. Can I just start a free trial instead? Absolutely. [Sign up for free](https://testingbot.com/users/sign_up) with no credit card required and explore TestingBot at your own pace. Book a demo whenever you have questions. --- URL: https://testingbot.com/resources # Resource Center Learn more about testing, test frameworks and the TestingBot service. 138 ## [Blog](https://testingbot.com/blog) News, releases and deep dives. Read → 55 ## [Articles](https://testingbot.com#section-articles) How-tos on Selenium, Cypress, Appium and more. Browse → 5 ## [Courses](https://testingbot.com#section-courses) Guided walk-throughs from setup to first test. Browse → 3 ## [Videos](https://testingbot.com#section-videos) Watch testing tutorials at your own pace. Browse → No resources match your search. ## Courses [All courses →](https://testingbot.com/resources/courses) [![Appium and WebDriverIO](https://testingbot.com/assets/resources/courses/3-16c2e4aeb71671a0c5f55a4e3a3904e094a9d0d4870edd12d4d3987785d46e9e.webp)](https://testingbot.com/resources/courses/appium-webdriverio "Appium and WebDriverIO") Course Feb 2023 ### [Appium and WebDriverIO](https://testingbot.com/resources/courses/appium-webdriverio) Learn what WebDriverIO is and how you can use it in combination with Appium to do automated testing. [Start course →](https://testingbot.com/resources/courses/appium-webdriverio) [![Local website testing](https://testingbot.com/assets/resources/courses/2-246ca55ba62d127afa39a405c4cbb8c5561a172f5f6f5b43d9ecbcc7ff5b3232.webp)](https://testingbot.com/resources/courses/local-website-testing "Local website testing") Course Jul 2022 ### [Local website testing](https://testingbot.com/resources/courses/local-website-testing) In this course we'll learn how to perform website testing against a website running on a local machine or private network. [Start course →](https://testingbot.com/resources/courses/local-website-testing) [![A Primer on Testing with TestingBot](https://testingbot.com/assets/resources/courses/1-8c736aa0fe0d0577791766e628cf8d20f291814618375635490323e249d980b0.webp)](https://testingbot.com/resources/courses/testing-with-testingbot "A Primer on Testing with TestingBot") Course Jul 2022 ### [A Primer on Testing with TestingBot](https://testingbot.com/resources/courses/testing-with-testingbot) Learn more on how to test with TestingBot [Start course →](https://testingbot.com/resources/courses/testing-with-testingbot) ## Articles [All articles →](https://testingbot.com/resources/articles) [![Parallel Testing: What Actually Scales](https://testingbot.com/assets/resources/articles/parallel-testing-32f51791d748c85621c4bf431293e46349cf0d81ec1a3262450b33f0edead4ef.webp)](https://testingbot.com/resources/articles/parallel-testing "Parallel Testing: What Actually Scales") Article Sep 1, 2026by Jochen Delabie ### [Parallel Testing: What Actually Scales](https://testingbot.com/resources/articles/parallel-testing) Why adding workers stops helping, how to shard so they are not idle, and what breaks the first time a suite runs concurrently. [Read article →](https://testingbot.com/resources/articles/parallel-testing) [![Automated Regression Testing in CI](https://testingbot.com/assets/resources/articles/automated-regression-testing-e283d0b6a88c787633d29f0cfb35cbb90a48136dace54720252843ebc4a36063.webp)](https://testingbot.com/resources/articles/automated-regression-testing "Automated Regression Testing in CI") Article Aug 31, 2026by Jochen Delabie ### [Automated Regression Testing in CI](https://testingbot.com/resources/articles/automated-regression-testing) How to build a regression suite that stays fast and trustworthy in CI, what to leave out of it, and when to run which part. [Read article →](https://testingbot.com/resources/articles/automated-regression-testing) [![Mobile App Test Automation at Scale](https://testingbot.com/assets/resources/articles/mobile-app-test-automation-12d7e7edc7c551f05282cd03ccd0de42da8e4c740d754286777972249471a36e.webp)](https://testingbot.com/resources/articles/mobile-app-test-automation "Mobile App Test Automation at Scale") Article Aug 31, 2026by Jochen Delabie ### [Mobile App Test Automation at Scale](https://testingbot.com/resources/articles/mobile-app-test-automation) How to choose between Appium, Espresso, XCUITest and Maestro, and what changes when you run the suite on real devices at scale. [Read article →](https://testingbot.com/resources/articles/mobile-app-test-automation) [![AI in Software Testing: What Works](https://testingbot.com/assets/resources/articles/ai-in-software-testing-ab8885a15af9f64731cdb97574153660eb0f409067995f0a089094cac76431e2.webp)](https://testingbot.com/resources/articles/ai-in-software-testing "AI in Software Testing: What Works") Article Aug 31, 2026by Jochen Delabie ### [AI in Software Testing: What Works](https://testingbot.com/resources/articles/ai-in-software-testing) A category-by-category look at where AI genuinely helps a test suite, where it does not, and how to evaluate a tool before you buy it. [Read article →](https://testingbot.com/resources/articles/ai-in-software-testing) [![Playwright MCP for Test Automation](https://testingbot.com/assets/resources/articles/playwright-mcp-da3126b2a5a5b0fe528ad6311ccb79e7a2943f7a3b637a5ae5e748087a59f39a.webp)](https://testingbot.com/resources/articles/playwright-mcp "Playwright MCP for Test Automation") Article Aug 31, 2026by Jochen Delabie ### [Playwright MCP for Test Automation](https://testingbot.com/resources/articles/playwright-mcp) What Playwright MCP does, how driving the accessibility tree differs from screenshot agents, and where it fits in a real test workflow. [Read article →](https://testingbot.com/resources/articles/playwright-mcp) [![Top Web Development Tools in 2025](https://testingbot.com/assets/resources/articles/55-68e8970c2d4f68ef0fe0f4f519991c5f4413c2c08802b5baba57baab219a239a.webp)](https://testingbot.com/resources/articles/web-development-tools "Top Web Development Tools in 2025") Article Sep 18, 2025by Jochen Delabie ### [Top Web Development Tools in 2025](https://testingbot.com/resources/articles/web-development-tools) The most popular web development tools in 2025, split into front-end, back-end and integrated, with what each one is actually good for. [Read article →](https://testingbot.com/resources/articles/web-development-tools) ## Videos [All videos →](https://testingbot.com/resources/videos) [![Selenium testing with Python Tutorial](https://testingbot.com/assets/videos/python-automated-bacffb33401f4772664d4d9c06455f2c0e4fabed0bfadf10770c0535a2bb6e76.png) ](https://testingbot.com/resources/videos/python-automated-testing-selenium "Selenium testing with Python Tutorial") Video Aug 19, 2021 ### [Selenium testing with Python Tutorial](https://testingbot.com/resources/videos/python-automated-testing-selenium) Want to use Python with Selenium? See this video to learn more. [Watch video →](https://testingbot.com/resources/videos/python-automated-testing-selenium) [![Automated Browser Testing with Ruby](https://testingbot.com/assets/videos/ruby-automated-a49902d15d265750dcd04b07a1fe142d8aac47976a9cae828e47835c01ce798f.png) ](https://testingbot.com/resources/videos/ruby-automated-testing-selenium "Automated Browser Testing with Ruby") Video Aug 19, 2021 ### [Automated Browser Testing with Ruby](https://testingbot.com/resources/videos/ruby-automated-testing-selenium) Video tutorial on how to run Ruby Automated Test on TestingBot. [Watch video →](https://testingbot.com/resources/videos/ruby-automated-testing-selenium) [![Automated testing with PHP and Selenium](https://testingbot.com/assets/videos/php-automated-29a925d6233fb94334c6c833b7e41d3e1ba05e04c5a01dc163a0fddac15d6889.png) ](https://testingbot.com/resources/videos/php-automated-testing-selenium "Automated testing with PHP and Selenium") Video Aug 19, 2021 ### [Automated testing with PHP and Selenium](https://testingbot.com/resources/videos/php-automated-testing-selenium) Learn how to use PHP with Selenium to do automated browser testing. [Watch video →](https://testingbot.com/resources/videos/php-automated-testing-selenium) ## From the blog [Read the blog →](https://testingbot.com/blog) [![Announcing AI Test Insights](https://testingbot.com/assets/blog/2026/ai-insights.jpg)](https://testingbot.com/blog/ai-test-insights "Announcing AI Test Insights") Blog Jun 04, 2026 ### [Announcing AI Test Insights](https://testingbot.com/blog/ai-test-insights) What is AI test failure analysis? Today we are introducing TestingBot AI Insights, our AI-powered test failure analysis feature. When an automated test fail... [Read post →](https://testingbot.com/blog/ai-test-insights) [![Selenium vs Playwright 2026](https://testingbot.com/assets/blog/2026/selenium-vs-playwright-2026.jpg)](https://testingbot.com/blog/selenium-vs-playwright "Selenium vs Playwright 2026") Blog May 21, 2026 ### [Selenium vs Playwright 2026](https://testingbot.com/blog/selenium-vs-playwright) Introduction: Why the Selenium vs Playwright Comparison Matters in 2026 The comparison of Selenium against Playwright has far transcended discussion threads... [Read post →](https://testingbot.com/blog/selenium-vs-playwright) [![Maestro Physical Device Testing](https://testingbot.com/assets/blog/2026/maestro-physical.jpg)](https://testingbot.com/blog/maestro-physical-device-testing "Maestro Physical Device Testing") Blog Jan 19, 2026 ### [Maestro Physical Device Testing](https://testingbot.com/blog/maestro-physical-device-testing) When we first introduced Maestro Cloud Testing on TestingBot, we focused on providing a reliable service for running Maestro cloud tests on emulators/si... [Read post →](https://testingbot.com/blog/maestro-physical-device-testing) --- URL: https://testingbot.com/resources/articles [Resources](https://testingbot.com/resources) / Articles # Articles How-tos on Selenium, Cypress, Appium, Playwright, Maestro and more. ## Featured articles ![mabl Alternative: TestingBot for Low-Code Automation](https://testingbot.com/assets/resources/articles/41-b22a387a55baec633cd8a35a021a538077099d0b28d8004d0e87f468dfc99017.webp) ### [mabl Alternative: TestingBot for Low-Code Automation](https://testingbot.com/resources/articles/mabl-alternative-testingbot) Convert Mabl tests to be used with TestingBot's cloud-based browser grid. Improve performance, test coverage and reliability. ![15 Best AI Testing Tools in 2026](https://testingbot.com/assets/resources/articles/44-0566068e7f37e24257a058aa6ff3398f11e192e89be40e10d2e8a81335d16b6c.webp) ### [15 Best AI Testing Tools in 2026](https://testingbot.com/resources/articles/top-15-ai-testing-tools) A reviewed list of 15 AI-driven testing tools for web and mobile: what each one actually automates, and where it fits alongside a normal test suite. ![Run Maestro tests in the cloud](https://testingbot.com/assets/resources/articles/45-54aab6a840d2561eab846e8bfde7ec26113ba601a9fcc15ddea883c2a80ec29c.webp) ### [Run Maestro tests in the cloud](https://testingbot.com/resources/articles/maestro-cloud-testing) What Maestro is, how its declarative flows differ from Appium, and how to run them against remote iOS and Android devices in the cloud. ![Javascript Test Frameworks: Top 15](https://testingbot.com/assets/resources/articles/37-807810cc7f5d4bcc71dae6cdb137625fe748e38e0194033318d7b269de8ef407.webp) ### [Javascript Test Frameworks: Top 15](https://testingbot.com/resources/articles/top-15-javascript-testing-frameworks) The 15 most popular JavaScript testing frameworks ranked, with the features, pros and cons of each, from Jest and Mocha through to Cypress. ![The best Python Web Testing frameworks](https://testingbot.com/assets/resources/articles/35-027df4e213761b025d3d612772682c8f7477b3adecaec14083f32c6c3bbec93d.webp) ### [The best Python Web Testing frameworks](https://testingbot.com/resources/articles/python-top-5-test-frameworks) Five Python test frameworks worth knowing, PyTest, unittest, Robot Framework, Behave and Nose, and how each one pairs with Selenium. ![BrowserStack Competitors: Sauce Labs, TestMu, TestingBot](https://testingbot.com/assets/resources/articles/cross-browser-testing-competitors-781173538787197ac315b04385ee92bb75177a1f4abdf93e1e1330a9ac0b90f1.webp) ### [BrowserStack Competitors: Sauce Labs, TestMu, TestingBot](https://testingbot.com/resources/articles/cross-browser-testing-competitors) A sourced, dated comparison of the four main cross browser testing clouds: what each costs to start, and what is bundled versus sold separately. All articles [Selenium](https://testingbot.com/resources/articles/topic/selenium) [Mobile App Testing](https://testingbot.com/resources/articles/topic/mobile-testing) [Test Frameworks](https://testingbot.com/resources/articles/topic/test-frameworks) [Web Testing](https://testingbot.com/resources/articles/topic/web-testing) [Playwright](https://testingbot.com/resources/articles/topic/playwright) [AI Testing](https://testingbot.com/resources/articles/topic/ai-testing) [Visual Testing](https://testingbot.com/resources/articles/topic/visual-testing) [Testing Practices](https://testingbot.com/resources/articles/topic/testing-practices) No articles match your filter. [![Parallel Testing: What Actually Scales](https://testingbot.com/assets/resources/articles/parallel-testing-32f51791d748c85621c4bf431293e46349cf0d81ec1a3262450b33f0edead4ef.webp)](https://testingbot.com/resources/articles/parallel-testing "Parallel Testing: What Actually Scales") Article Sep 1, 2026by Jochen Delabie ## [Parallel Testing: What Actually Scales](https://testingbot.com/resources/articles/parallel-testing) Why adding workers stops helping, how to shard so they are not idle, and what breaks the first time a suite runs concurrently. [Read article →](https://testingbot.com/resources/articles/parallel-testing) [![Automated Regression Testing in CI](https://testingbot.com/assets/resources/articles/automated-regression-testing-e283d0b6a88c787633d29f0cfb35cbb90a48136dace54720252843ebc4a36063.webp)](https://testingbot.com/resources/articles/automated-regression-testing "Automated Regression Testing in CI") Article Aug 31, 2026by Jochen Delabie ## [Automated Regression Testing in CI](https://testingbot.com/resources/articles/automated-regression-testing) How to build a regression suite that stays fast and trustworthy in CI, what to leave out of it, and when to run which part. [Read article →](https://testingbot.com/resources/articles/automated-regression-testing) [![Mobile App Test Automation at Scale](https://testingbot.com/assets/resources/articles/mobile-app-test-automation-12d7e7edc7c551f05282cd03ccd0de42da8e4c740d754286777972249471a36e.webp)](https://testingbot.com/resources/articles/mobile-app-test-automation "Mobile App Test Automation at Scale") Article Aug 31, 2026by Jochen Delabie ## [Mobile App Test Automation at Scale](https://testingbot.com/resources/articles/mobile-app-test-automation) How to choose between Appium, Espresso, XCUITest and Maestro, and what changes when you run the suite on real devices at scale. [Read article →](https://testingbot.com/resources/articles/mobile-app-test-automation) [![AI in Software Testing: What Works](https://testingbot.com/assets/resources/articles/ai-in-software-testing-ab8885a15af9f64731cdb97574153660eb0f409067995f0a089094cac76431e2.webp)](https://testingbot.com/resources/articles/ai-in-software-testing "AI in Software Testing: What Works") Article Aug 31, 2026by Jochen Delabie ## [AI in Software Testing: What Works](https://testingbot.com/resources/articles/ai-in-software-testing) A category-by-category look at where AI genuinely helps a test suite, where it does not, and how to evaluate a tool before you buy it. [Read article →](https://testingbot.com/resources/articles/ai-in-software-testing) [![Playwright MCP for Test Automation](https://testingbot.com/assets/resources/articles/playwright-mcp-da3126b2a5a5b0fe528ad6311ccb79e7a2943f7a3b637a5ae5e748087a59f39a.webp)](https://testingbot.com/resources/articles/playwright-mcp "Playwright MCP for Test Automation") Article Aug 31, 2026by Jochen Delabie ## [Playwright MCP for Test Automation](https://testingbot.com/resources/articles/playwright-mcp) What Playwright MCP does, how driving the accessibility tree differs from screenshot agents, and where it fits in a real test workflow. [Read article →](https://testingbot.com/resources/articles/playwright-mcp) [![Top Web Development Tools in 2025](https://testingbot.com/assets/resources/articles/55-68e8970c2d4f68ef0fe0f4f519991c5f4413c2c08802b5baba57baab219a239a.webp)](https://testingbot.com/resources/articles/web-development-tools "Top Web Development Tools in 2025") Article Sep 18, 2025by Jochen Delabie ## [Top Web Development Tools in 2025](https://testingbot.com/resources/articles/web-development-tools) The most popular web development tools in 2025, split into front-end, back-end and integrated, with what each one is actually good for. [Read article →](https://testingbot.com/resources/articles/web-development-tools) [![Most popular Screen Resolutions](https://testingbot.com/assets/resources/articles/54-5f84cd0ac4f2b96527545559c41f9e198b9bcb433dc32d6f58e33a4088ba4be1.webp)](https://testingbot.com/resources/articles/common-screen-resolutions "Most popular Screen Resolutions") Article Jun 22, 2025by Jochen Delabie ## [Most popular Screen Resolutions](https://testingbot.com/resources/articles/common-screen-resolutions) The most common desktop, mobile and tablet screen resolutions in 2025, and what they mean for responsive design and the breakpoints you test. [Read article →](https://testingbot.com/resources/articles/common-screen-resolutions) [![Selenium Python Tutorial](https://testingbot.com/assets/resources/articles/51-39337d128e18d16ddec5fe39124c2302f334e0f21e06f361cfd19e0b719e8861.webp)](https://testingbot.com/resources/articles/python-selenium-web-automation-test "Selenium Python Tutorial") Article May 26, 2025by Jochen Delabie ## [Selenium Python Tutorial](https://testingbot.com/resources/articles/python-selenium-web-automation-test) Getting started with Selenium WebDriver in Python: a first test, locating elements, implicit and explicit waits, alerts and teardown. [Read article →](https://testingbot.com/resources/articles/python-selenium-web-automation-test) [![How to Change Browser Settings](https://testingbot.com/assets/resources/articles/52-5bb27f13382030a8890a4a9c2d53a015fac4512df32e6a633d59a27c6ccd71cf.webp)](https://testingbot.com/resources/articles/how-to-change-browser-settings "How to Change Browser Settings") Article May 22, 2025by Jochen Delabie ## [How to Change Browser Settings](https://testingbot.com/resources/articles/how-to-change-browser-settings) Step-by-step instructions for changing proxy, cookie and cache settings in Chrome, Edge and Firefox, on both Windows and macOS. [Read article →](https://testingbot.com/resources/articles/how-to-change-browser-settings) [![What is isolation testing?](https://testingbot.com/assets/resources/articles/50-6cf90dd5ec7b54f52a74dd2870938d81958ac171ec40d066cb89de674bdc11a6.webp)](https://testingbot.com/resources/articles/isolation-test "What is isolation testing?") Article May 17, 2025by Jochen Delabie ## [What is isolation testing?](https://testingbot.com/resources/articles/isolation-test) Isolation Testing means verifying the functionality of a specific part of software on its own, free from outside influences. [Read article →](https://testingbot.com/resources/articles/isolation-test) [![Why is accessibility testing important in web design?](https://testingbot.com/assets/resources/articles/49-ddec8d6d134f657a98d9537849a0bb94fe908d49159be466aa83da15760120a4.webp)](https://testingbot.com/resources/articles/why-accessibility-testing "Why is accessibility testing important in web design?") Article May 7, 2025by Jochen Delabie ## [Why is accessibility testing important in web design?](https://testingbot.com/resources/articles/why-accessibility-testing) Why accessible design matters, what the four POUR principles ask of you, and how to check a site against them as part of normal testing. [Read article →](https://testingbot.com/resources/articles/why-accessibility-testing) [![Protect your Shopify store](https://testingbot.com/assets/resources/articles/47-93b1b2d2876da23984eced6f17b1690865746b54eaa11b0e9cbc17c9b0219e2d.webp)](https://testingbot.com/resources/articles/shopify-store-monitoring "Protect your Shopify store") Article Mar 14, 2025by Jochen Delabie ## [Protect your Shopify store](https://testingbot.com/resources/articles/shopify-store-monitoring) The failures that quietly cost a Shopify store revenue, from slow loads to a broken checkout, and how scheduled monitoring catches them first. [Read article →](https://testingbot.com/resources/articles/shopify-store-monitoring) [![15 Best AI Testing Tools in 2026](https://testingbot.com/assets/resources/articles/44-0566068e7f37e24257a058aa6ff3398f11e192e89be40e10d2e8a81335d16b6c.webp)](https://testingbot.com/resources/articles/top-15-ai-testing-tools "15 Best AI Testing Tools in 2026") Article Mar 13, 2025by Jochen Delabie ## [15 Best AI Testing Tools in 2026](https://testingbot.com/resources/articles/top-15-ai-testing-tools) A reviewed list of 15 AI-driven testing tools for web and mobile: what each one actually automates, and where it fits alongside a normal test suite. [Read article →](https://testingbot.com/resources/articles/top-15-ai-testing-tools) [![Run Maestro tests in the cloud](https://testingbot.com/assets/resources/articles/45-54aab6a840d2561eab846e8bfde7ec26113ba601a9fcc15ddea883c2a80ec29c.webp)](https://testingbot.com/resources/articles/maestro-cloud-testing "Run Maestro tests in the cloud") Article Dec 18, 2024by Jochen Delabie ## [Run Maestro tests in the cloud](https://testingbot.com/resources/articles/maestro-cloud-testing) What Maestro is, how its declarative flows differ from Appium, and how to run them against remote iOS and Android devices in the cloud. [Read article →](https://testingbot.com/resources/articles/maestro-cloud-testing) [![Handling Exceptions with Selenium Webdriver](https://testingbot.com/assets/resources/articles/43-044fb4e2070e941fec050b38f1c4ed34a06e7c8b63c67a6b8758805869eb75be.webp)](https://testingbot.com/resources/articles/exception-handling-with-selenium "Handling Exceptions with Selenium Webdriver") Article Nov 29, 2024by Jochen Delabie ## [Handling Exceptions with Selenium Webdriver](https://testingbot.com/resources/articles/exception-handling-with-selenium) The Selenium WebDriver exceptions you will actually hit, from NoSuchElement to StaleElementReference, what triggers each and how to prevent them. [Read article →](https://testingbot.com/resources/articles/exception-handling-with-selenium) [![How to write a Test Strategy Document?](https://testingbot.com/assets/resources/articles/42-a0cdc47f1faf6944ef5a5e9c0f5ba271b68a4e70aa101625f9a8421c1519ab3a.webp)](https://testingbot.com/resources/articles/how-to-write-test-strategy-document "How to write a Test Strategy Document?") Article Sep 13, 2024by Jochen Delabie ## [How to write a Test Strategy Document?](https://testingbot.com/resources/articles/how-to-write-test-strategy-document) Learn how to create a test strategy document. This is a document that outlines what needs to be tested at which stage of the build process. [Read article →](https://testingbot.com/resources/articles/how-to-write-test-strategy-document) [![mabl Alternative: TestingBot for Low-Code Automation](https://testingbot.com/assets/resources/articles/41-b22a387a55baec633cd8a35a021a538077099d0b28d8004d0e87f468dfc99017.webp)](https://testingbot.com/resources/articles/mabl-alternative-testingbot "mabl Alternative: TestingBot for Low-Code Automation") Article Aug 12, 2024by Jochen Delabie ## [mabl Alternative: TestingBot for Low-Code Automation](https://testingbot.com/resources/articles/mabl-alternative-testingbot) Convert Mabl tests to be used with TestingBot's cloud-based browser grid. Improve performance, test coverage and reliability. [Read article →](https://testingbot.com/resources/articles/mabl-alternative-testingbot) [![Fake Webcam and Microphone in Chrome for Selenium Tests](https://testingbot.com/assets/resources/articles/40-6133f19cdc01c7d6461f3f138789185ede755adf911eb2da670961ba41fcce30.webp)](https://testingbot.com/resources/articles/fake-webcam-microphone-chrome "Fake Webcam and Microphone in Chrome for Selenium Tests") Article Jun 27, 2024by Jochen Delabie ## [Fake Webcam and Microphone in Chrome for Selenium Tests](https://testingbot.com/resources/articles/fake-webcam-microphone-chrome) How to feed Chrome a fake webcam and microphone during automated tests, so Selenium, Puppeteer, Playwright and Cypress can drive media features. [Read article →](https://testingbot.com/resources/articles/fake-webcam-microphone-chrome) [![Javascript Test Frameworks: Top 15](https://testingbot.com/assets/resources/articles/37-807810cc7f5d4bcc71dae6cdb137625fe748e38e0194033318d7b269de8ef407.webp)](https://testingbot.com/resources/articles/top-15-javascript-testing-frameworks "Javascript Test Frameworks: Top 15") Article Jun 13, 2024by Jochen Delabie ## [Javascript Test Frameworks: Top 15](https://testingbot.com/resources/articles/top-15-javascript-testing-frameworks) The 15 most popular JavaScript testing frameworks ranked, with the features, pros and cons of each, from Jest and Mocha through to Cypress. [Read article →](https://testingbot.com/resources/articles/top-15-javascript-testing-frameworks) [![Automated Browser Extension Testing](https://testingbot.com/assets/resources/articles/36-0533b9ac97e8fa44703058e149be135278cb5e1a9e29c273eb37b7c70c47c973.webp)](https://testingbot.com/resources/articles/automated-browser-extension-testing "Automated Browser Extension Testing") Article Feb 21, 2024by Jochen Delabie ## [Automated Browser Extension Testing](https://testingbot.com/resources/articles/automated-browser-extension-testing) How to install and drive a browser extension end to end, with working examples for Selenium, Puppeteer and Playwright on the major browsers. [Read article →](https://testingbot.com/resources/articles/automated-browser-extension-testing) [![The best Python Web Testing frameworks](https://testingbot.com/assets/resources/articles/35-027df4e213761b025d3d612772682c8f7477b3adecaec14083f32c6c3bbec93d.webp)](https://testingbot.com/resources/articles/python-top-5-test-frameworks "The best Python Web Testing frameworks") Article Feb 10, 2024by Jochen Delabie ## [The best Python Web Testing frameworks](https://testingbot.com/resources/articles/python-top-5-test-frameworks) Five Python test frameworks worth knowing, PyTest, unittest, Robot Framework, Behave and Nose, and how each one pairs with Selenium. [Read article →](https://testingbot.com/resources/articles/python-top-5-test-frameworks) [![Cucumber testing with Selenium and IntelliJ or Eclipse](https://testingbot.com/assets/resources/articles/34-aa3115db36ed77849b9407f6d941da71f058c07bcee8a0fe4e2c19fd59726ba4.webp)](https://testingbot.com/resources/articles/cucumber-eclipse-intellij "Cucumber testing with Selenium and IntelliJ or Eclipse") Article Nov 10, 2023by Jochen Delabie ## [Cucumber testing with Selenium and IntelliJ or Eclipse](https://testingbot.com/resources/articles/cucumber-eclipse-intellij) How to run Cucumber and Selenium tests straight from IntelliJ or Eclipse, with worked examples for both IDEs and for Cucumber in JavaScript. [Read article →](https://testingbot.com/resources/articles/cucumber-eclipse-intellij) [![ElementClickInterceptedException in Selenium](https://testingbot.com/assets/resources/articles/33-47ac7bd953d8c5e5d1274bfe324e755e37c21003cf9224b826e4833a9f1e9001.webp)](https://testingbot.com/resources/articles/selenium-elementclickinterceptedexception "ElementClickInterceptedException in Selenium") Article Sep 26, 2023by Jochen Delabie ## [ElementClickInterceptedException in Selenium](https://testingbot.com/resources/articles/selenium-elementclickinterceptedexception) What causes an ElementClickInterceptedException in Selenium, how it differs from the other click failures, and how to handle it reliably. [Read article →](https://testingbot.com/resources/articles/selenium-elementclickinterceptedexception) [![Playwright Codegen: Record Tests in VS Code](https://testingbot.com/assets/resources/articles/32-277318b88c5e35eb2c96f03d9b53a1ef8eb2d96db184679892b5008b0bad9d0f.webp)](https://testingbot.com/resources/articles/playwright-recorder "Playwright Codegen: Record Tests in VS Code") Article Sep 20, 2023by Jochen Delabie ## [Playwright Codegen: Record Tests in VS Code](https://testingbot.com/resources/articles/playwright-recorder) How to record Playwright scripts from Visual Studio Code rather than writing them by hand, then run the recorded tests on a browser grid. [Read article →](https://testingbot.com/resources/articles/playwright-recorder) [![Testing with React and Selenium](https://testingbot.com/assets/resources/articles/31-744de7cfd83a956722f63fdccfc836d4cd8ee5182785fd84f7bbabe3a40e1615.webp)](https://testingbot.com/resources/articles/react-selenium-testing "Testing with React and Selenium") Article Sep 6, 2023by Jochen Delabie ## [Testing with React and Selenium](https://testingbot.com/resources/articles/react-selenium-testing) How to test a React site with both React Testing Library and Selenium WebDriver, and when each of the two is the right tool to reach for. [Read article →](https://testingbot.com/resources/articles/react-selenium-testing) [![How to fix Flaky Tests](https://testingbot.com/assets/resources/articles/30-da23ef05df0b04704e23e77b3b068cc04d038f42ca12ae7123c8039acf5fb41d.webp)](https://testingbot.com/resources/articles/handling-test-flakiness "How to fix Flaky Tests") Article Aug 25, 2023by Jochen Delabie ## [How to fix Flaky Tests](https://testingbot.com/resources/articles/handling-test-flakiness) Why a test passes most of the time and fails the rest, with concrete fixes for flaky Selenium, Appium, Playwright and Puppeteer runs. [Read article →](https://testingbot.com/resources/articles/handling-test-flakiness) [![Test Automation with ChatGPT](https://testingbot.com/assets/resources/articles/29-7c01b99e7284a560ad99094e3ce738c4942eee113a51295a0a8e9cd8fe0cbba5.webp)](https://testingbot.com/resources/articles/test-automation-with-chatgpt "Test Automation with ChatGPT") Article Aug 14, 2023by Jochen Delabie ## [Test Automation with ChatGPT](https://testingbot.com/resources/articles/test-automation-with-chatgpt) How to prompt ChatGPT into generating Selenium, Appium, Puppeteer and Playwright tests, and how far the output can be trusted as written. [Read article →](https://testingbot.com/resources/articles/test-automation-with-chatgpt) [![Selenium and Generative AI](https://testingbot.com/assets/resources/articles/28-af707bf052844eb00f861bb6ebba57e6be4479dde1c78a5df4e70d2f597a8b18.webp)](https://testingbot.com/resources/articles/generative-ai-selenium "Selenium and Generative AI") Article Aug 6, 2023by Jochen Delabie ## [Selenium and Generative AI](https://testingbot.com/resources/articles/generative-ai-selenium) How to use generative AI to produce realistic test data for Selenium runs, and why varied data uncovers bugs that fixed fixtures never will. [Read article →](https://testingbot.com/resources/articles/generative-ai-selenium) [![Appium 2 and Appium 3 Migration Guide](https://testingbot.com/assets/resources/articles/27-121da26036885301ba438e1cdfc81e7ae8045cbc53c0dafaff5b4436000590b1.webp)](https://testingbot.com/resources/articles/appium-2-migration "Appium 2 and Appium 3 Migration Guide") Article Jul 27, 2023by Jochen Delabie ## [Appium 2 and Appium 3 Migration Guide](https://testingbot.com/resources/articles/appium-2-migration) What changed between Appium 1, 2 and 3: drivers, capabilities, the removed touch endpoints, and how to upgrade an existing suite. [Read article →](https://testingbot.com/resources/articles/appium-2-migration) [![Testing Tailwind CSS webpages](https://testingbot.com/assets/resources/articles/26-90e656f61416f2cfe616ab4bca7b673554e4fd05702863d3274ac8ab06a00c95.webp)](https://testingbot.com/resources/articles/tailwind-testing "Testing Tailwind CSS webpages") Article Jul 26, 2023by Jochen Delabie ## [Testing Tailwind CSS webpages](https://testingbot.com/resources/articles/tailwind-testing) How to write automated tests against a Tailwind CSS site when there are no semantic class names, and how to catch visual regressions. [Read article →](https://testingbot.com/resources/articles/tailwind-testing) [![Automate native iOS Apps with XCUITest](https://testingbot.com/assets/resources/articles/25-bb2ccb531384c461ef792214839e9e8f99a4b08b1293730ceeab67e4a8352563.webp)](https://testingbot.com/resources/articles/automate-native-ios-apps-xcuitest "Automate native iOS Apps with XCUITest") Article Jul 22, 2023by Jochen Delabie ## [Automate native iOS Apps with XCUITest](https://testingbot.com/resources/articles/automate-native-ios-apps-xcuitest) How to automate Apple's own iOS apps with XCUITest, with code for driving Settings, Messages and Photos, and how to extend it to other apps. [Read article →](https://testingbot.com/resources/articles/automate-native-ios-apps-xcuitest) [![Using Touch Actions with Appium](https://testingbot.com/assets/resources/articles/24-490f70ad8a80e4534523707125b88e44e70fe19b126be4f44e689bbd5f6a0cbf.webp)](https://testingbot.com/resources/articles/touch-actions-appium "Using Touch Actions with Appium") Article Jul 18, 2023by Jochen Delabie ## [Using Touch Actions with Appium](https://testingbot.com/resources/articles/touch-actions-appium) How to simulate taps, swipes and pinches on real devices with Appium Touch Actions, with examples, best practices and common pitfalls. [Read article →](https://testingbot.com/resources/articles/touch-actions-appium) [![How to Inspect Element using UIAutomatorViewer](https://testingbot.com/assets/resources/articles/23-d6314d62f1aa6a6c51fb960d1b65bc3aeb14ee0ade20527f6546bfd5777536d2.webp)](https://testingbot.com/resources/articles/android-uiautomatorviewer "How to Inspect Element using UIAutomatorViewer") Article Jul 14, 2023by Jochen Delabie ## [How to Inspect Element using UIAutomatorViewer](https://testingbot.com/resources/articles/android-uiautomatorviewer) How to use UIAutomatorViewer from the Android SDK to inspect app screens and find the selectors your automated Android tests need. [Read article →](https://testingbot.com/resources/articles/android-uiautomatorviewer) [![Dark Mode Testing with Appium](https://testingbot.com/assets/resources/articles/22-ed15fa0f3fde7566416fff5078b44689dc274439b2fb829f8c68603e909a65d6.webp)](https://testingbot.com/resources/articles/dark-mode-testing "Dark Mode Testing with Appium") Article Apr 6, 2023by Jochen Delabie ## [Dark Mode Testing with Appium](https://testingbot.com/resources/articles/dark-mode-testing) How to switch a native app between light and dark mode from an Appium test on iOS and Android, and the layout bugs dark mode tends to expose. [Read article →](https://testingbot.com/resources/articles/dark-mode-testing) [![Angular UI Testing](https://testingbot.com/assets/resources/articles/21-15af461a46c449ef26dfb952ed9d0fc327c97d9ca815f24f8fe9ee60b9aa64a5.webp)](https://testingbot.com/resources/articles/angular-ui-testing "Angular UI Testing") Article Mar 26, 2023by Jochen Delabie ## [Angular UI Testing](https://testingbot.com/resources/articles/angular-ui-testing) How to add visual regression testing to an Angular app with Cypress, comparing screenshots against a baseline to catch unintended UI changes. [Read article →](https://testingbot.com/resources/articles/angular-ui-testing) [![Visual Regression Testing with Python](https://testingbot.com/assets/resources/articles/20-823a333072d87c93f6d75a502e3073e32bd0bee1d3142540ce0b42da6fa7ccf0.webp)](https://testingbot.com/resources/articles/python-visual-testing "Visual Regression Testing with Python") Article Mar 21, 2023by Jochen Delabie ## [Visual Regression Testing with Python](https://testingbot.com/resources/articles/python-visual-testing) How to add visual regression testing to a Python suite: creating a baseline, comparing against it, and keeping screenshots stable over time. [Read article →](https://testingbot.com/resources/articles/python-visual-testing) [![Android Espresso Tutorial](https://testingbot.com/assets/resources/articles/19-fe4971eb8049097c25f2cc2fce95b71b94f3bb342e094c219b2404c09374f015.webp)](https://testingbot.com/resources/articles/android-espresso-testing "Android Espresso Tutorial") Article Jul 1, 2022by Jochen Delabie ## [Android Espresso Tutorial](https://testingbot.com/resources/articles/android-espresso-testing) Why Espresso suits Android UI testing, how to add it to a project and write a first test, and how it compares with the alternatives. [Read article →](https://testingbot.com/resources/articles/android-espresso-testing) [![Appium Page Object Model & Page Factory](https://testingbot.com/assets/resources/articles/18-d5ebe90196feee55fbf779752565bb813e5ff45ba7329882b5bdd38225f35581.webp)](https://testingbot.com/resources/articles/page-object-model-appium "Appium Page Object Model & Page Factory") Article Jun 23, 2022by Jochen Delabie ## [Appium Page Object Model & Page Factory](https://testingbot.com/resources/articles/page-object-model-appium) How the Page Object Model keeps Appium tests readable, with a worked example, the Page Factory, @FindBy locators and when to cache lookups. [Read article →](https://testingbot.com/resources/articles/page-object-model-appium) [![Getting Started with Appium](https://testingbot.com/assets/resources/articles/17-0f1ccf4b98b8ad7a8deb4040a2d3f04ef1131418471be27b1c7d0b1b758880eb.webp)](https://testingbot.com/resources/articles/getting-started-with-appium "Getting Started with Appium") Article May 27, 2022by Jochen Delabie ## [Getting Started with Appium](https://testingbot.com/resources/articles/getting-started-with-appium) Getting started with Appium for native and hybrid app testing, with performance tips and how to troubleshoot using Appium Inspector. [Read article →](https://testingbot.com/resources/articles/getting-started-with-appium) [![Getting Started with IntelliJ and Selenium WebDriver](https://testingbot.com/assets/resources/articles/16-511fdda6b13b4e7660736fb97eb23be9b62874f39294870fb66212303d530cca.webp)](https://testingbot.com/resources/articles/getting-started-with-webdriver-in-java-using-intellij "Getting Started with IntelliJ and Selenium WebDriver") Article Mar 3, 2022by Jochen Delabie ## [Getting Started with IntelliJ and Selenium WebDriver](https://testingbot.com/resources/articles/getting-started-with-webdriver-in-java-using-intellij) How to set up IntelliJ IDEA for Selenium WebDriver: the project, browser drivers, JUnit, and running the resulting tests in the cloud. [Read article →](https://testingbot.com/resources/articles/getting-started-with-webdriver-in-java-using-intellij) [![Internationalization testing of websites and mobile apps](https://testingbot.com/assets/resources/articles/15-4127c39c1b9ffddd30ae0a465903d841df9fbbd87f39dbef0f260ab101340123.webp)](https://testingbot.com/resources/articles/internationalization-testing "Internationalization testing of websites and mobile apps") Article Feb 23, 2022by Jochen Delabie ## [Internationalization testing of websites and mobile apps](https://testingbot.com/resources/articles/internationalization-testing) Learn more about Internationalization Testing: how to prepare and test your website or mobile app for different regions in the world. [Read article →](https://testingbot.com/resources/articles/internationalization-testing) [![Visual Testing with Playwright](https://testingbot.com/assets/resources/articles/14-f3f2e8954c8a47cf0bb29059b44f4d2f40998442e22db6f8acdccb8e5b34d8c1.webp)](https://testingbot.com/resources/articles/playwright-visual-regression-testing "Visual Testing with Playwright") Article Dec 28, 2021by Jochen Delabie ## [Visual Testing with Playwright](https://testingbot.com/resources/articles/playwright-visual-regression-testing) Playwright provides automated browser testing. It offers a built-in feature to perform visual regression testing for your website. [Read article →](https://testingbot.com/resources/articles/playwright-visual-regression-testing) [![Appium Flutter Driver: Testing Flutter Apps](https://testingbot.com/assets/resources/articles/13-77f190ec948df41b31594f8fc224348844ebc412f374eb6f18c9d4215b4fa49b.webp)](https://testingbot.com/resources/articles/appium-flutter-app-testing "Appium Flutter Driver: Testing Flutter Apps") Article Nov 16, 2021by Jochen Delabie ## [Appium Flutter Driver: Testing Flutter Apps](https://testingbot.com/resources/articles/appium-flutter-app-testing) How to test a Flutter app with Appium: what the framework compiles to, how the Appium Flutter Driver works, and how to build an app for testing. [Read article →](https://testingbot.com/resources/articles/appium-flutter-app-testing) [![How to do localisation testing for mobile apps](https://testingbot.com/assets/resources/articles/12-9db6c7be671827ade1212fa6e476ee36e3365f01569100ee46fe22405c035760.webp)](https://testingbot.com/resources/articles/localized-testing-mobile-app "How to do localisation testing for mobile apps") Article Nov 10, 2021by Jochen Delabie ## [How to do localisation testing for mobile apps](https://testingbot.com/resources/articles/localized-testing-mobile-app) Find out how to perform localisation testing with mobile apps. Change your language or locale with Appium, XCUITest and Espresso. [Read article →](https://testingbot.com/resources/articles/localized-testing-mobile-app) [![Automated Testing with Puppeteer](https://testingbot.com/assets/resources/articles/11-416015eb394d928299663409dccd67993b0a664f4b3ff456803fefc6bf1f325a.webp)](https://testingbot.com/resources/articles/automated-testing-with-puppeteer "Automated Testing with Puppeteer") Article Sep 8, 2021by Jochen Delabie ## [Automated Testing with Puppeteer](https://testingbot.com/resources/articles/automated-testing-with-puppeteer) Puppeteer combined with a test framework provides a great way to run automated browser tests. Follow this guide for more information. [Read article →](https://testingbot.com/resources/articles/automated-testing-with-puppeteer) [![Tutorial on debugging Cypress tests](https://testingbot.com/assets/resources/articles/10-7ae9f6d9435bea1c54ec815ad535dc4a590086e6165263a9eba3441635d55726.webp)](https://testingbot.com/resources/articles/how-to-debug-cypress-tests "Tutorial on debugging Cypress tests") Article Sep 6, 2021by Jochen Delabie ## [Tutorial on debugging Cypress tests](https://testingbot.com/resources/articles/how-to-debug-cypress-tests) How to debug a failing Cypress test using the debugger, console logs, the stack trace, and the debug and pause commands. [Read article →](https://testingbot.com/resources/articles/how-to-debug-cypress-tests) [![How to Handle Cookies in Selenium WebDriver](https://testingbot.com/assets/resources/articles/9-5a522a91a2140dbb74c2c0b484c669aed90159647c66f1ab98cb1b11579fea6e.webp)](https://testingbot.com/resources/articles/handling-cookies-with-selenium "How to Handle Cookies in Selenium WebDriver") Article Aug 27, 2021by Jochen Delabie ## [How to Handle Cookies in Selenium WebDriver](https://testingbot.com/resources/articles/handling-cookies-with-selenium) Handling cookies with Selenium WebDriver is a common task, since most websites use cookies. In this guide, we'll show you how to do this with Selenium. [Read article →](https://testingbot.com/resources/articles/handling-cookies-with-selenium) [![How to use the Actions Class In Selenium](https://testingbot.com/assets/resources/articles/8-f84768dacc47dbc268338db641c2c210f5da0b9d258f4ea8592d298b015473bf.webp)](https://testingbot.com/resources/articles/what-are-selenium-actions "How to use the Actions Class In Selenium") Article Aug 25, 2021by Jochen Delabie ## [How to use the Actions Class In Selenium](https://testingbot.com/resources/articles/what-are-selenium-actions) Selenium WebDriver comes with an Action Class, which allows you to simulate user input events, such as mouse and keyboard actions. [Read article →](https://testingbot.com/resources/articles/what-are-selenium-actions) [![Handling Captcha during Automated Testing](https://testingbot.com/assets/resources/articles/7-b54c5d487c7cc09e40ed52a5c59ecc06e26e172e66e215a8f7fb3e6c7d1c1de3.webp)](https://testingbot.com/resources/articles/automated-testing-with-captcha "Handling Captcha during Automated Testing") Article Aug 24, 2021by Jochen Delabie ## [Handling Captcha during Automated Testing](https://testingbot.com/resources/articles/automated-testing-with-captcha) Practical ways to get an automated test past a captcha: disable it in staging, click the reCaptcha checkbox, or solve it by hand mid-run. [Read article →](https://testingbot.com/resources/articles/automated-testing-with-captcha) [![Cypress and Cucumber Browser Testing](https://testingbot.com/assets/resources/articles/6-495c5e14cb4bf1e282858fc7e2cfcbe89775e3790b88d34c15bf9a6f36d387d4.webp)](https://testingbot.com/resources/articles/cypress-cucumber-testing "Cypress and Cucumber Browser Testing") Article Aug 20, 2021by Jochen Delabie ## [Cypress and Cucumber Browser Testing](https://testingbot.com/resources/articles/cypress-cucumber-testing) How to combine Cypress with Cucumber so end-to-end tests read as plain language, including a worked example and the TypeScript setup. [Read article →](https://testingbot.com/resources/articles/cypress-cucumber-testing) [![TestNG automation with Selenium](https://testingbot.com/assets/resources/articles/5-46a5444bcdd6b07d6bbe8e9e58be5093fe3aaa8d1a7939b89291d7173d4e4c74.webp)](https://testingbot.com/resources/articles/testng-selenium "TestNG automation with Selenium") Article Aug 19, 2021by Jochen Delabie ## [TestNG automation with Selenium](https://testingbot.com/resources/articles/testng-selenium) TestNG is a popular Java Test Framework which allows you to write clean Selenium tests with built-in parallelisation. [Read article →](https://testingbot.com/resources/articles/testng-selenium) [![BrowserStack Competitors: Sauce Labs, TestMu, TestingBot](https://testingbot.com/assets/resources/articles/cross-browser-testing-competitors-781173538787197ac315b04385ee92bb75177a1f4abdf93e1e1330a9ac0b90f1.webp)](https://testingbot.com/resources/articles/cross-browser-testing-competitors "BrowserStack Competitors: Sauce Labs, TestMu, TestingBot") Article Aug 17, 2021by Jochen Delabie ## [BrowserStack Competitors: Sauce Labs, TestMu, TestingBot](https://testingbot.com/resources/articles/cross-browser-testing-competitors) A sourced, dated comparison of the four main cross browser testing clouds: what each costs to start, and what is bundled versus sold separately. [Read article →](https://testingbot.com/resources/articles/cross-browser-testing-competitors) [![Why Automated Testing Matters](https://testingbot.com/assets/resources/articles/automated-testing-69222250f39d63330f8c3e20dd9cb0d351a002a1a285b926de7b824e6b5f18e7.webp)](https://testingbot.com/resources/articles/automated-testing "Why Automated Testing Matters") Article Aug 17, 2021by Jochen Delabie ## [Why Automated Testing Matters](https://testingbot.com/resources/articles/automated-testing) Why manual checking stops scaling across browsers and devices, where Selenium fits, and what a hosted grid changes. [Read article →](https://testingbot.com/resources/articles/automated-testing) Article Aug 17, 2021by Jochen Delabie ## [Why Automated Testing Matters](https://testingbot.com/resources/articles/why-automated-testing) Why verifying a site by hand across every browser and device stops scaling, and what automated testing catches that manual checking misses. [Read article →](https://testingbot.com/resources/articles/why-automated-testing) Article Aug 17, 2021by Jochen Delabie ## [Why use TestingBot - a Selenium grid in the cloud](https://testingbot.com/resources/articles/why-use-testingbot) So you are looking to run Selenium tests on multiple browsers and want to find out more information on how to run these tests. [Read article →](https://testingbot.com/resources/articles/why-use-testingbot) --- URL: https://testingbot.com/resources/courses [Resources](https://testingbot.com/resources) / Courses # Courses Follow courses and tutorials for Selenium, Appium, Cypress, Espresso and XCUITest. No courses match your filter. [![Appium and WebDriverIO](https://testingbot.com/assets/resources/courses/3-16c2e4aeb71671a0c5f55a4e3a3904e094a9d0d4870edd12d4d3987785d46e9e.webp)](https://testingbot.com/resources/courses/appium-webdriverio "Appium and WebDriverIO") Course Feb 2023 ## [Appium and WebDriverIO](https://testingbot.com/resources/courses/appium-webdriverio) Learn what WebDriverIO is and how you can use it in combination with Appium to do automated testing. - **Duration:** 10 minutes - **Difficulty:** Easy [Start course →](https://testingbot.com/resources/courses/appium-webdriverio) [![Local website testing](https://testingbot.com/assets/resources/courses/2-246ca55ba62d127afa39a405c4cbb8c5561a172f5f6f5b43d9ecbcc7ff5b3232.webp)](https://testingbot.com/resources/courses/local-website-testing "Local website testing") Course Jul 2022 ## [Local website testing](https://testingbot.com/resources/courses/local-website-testing) In this course we'll learn how to perform website testing against a website running on a local machine or private network. - **Duration:** 12 minutes - **Difficulty:** Medium [Start course →](https://testingbot.com/resources/courses/local-website-testing) [![A Primer on Testing with TestingBot](https://testingbot.com/assets/resources/courses/1-8c736aa0fe0d0577791766e628cf8d20f291814618375635490323e249d980b0.webp)](https://testingbot.com/resources/courses/testing-with-testingbot "A Primer on Testing with TestingBot") Course Jul 2022 ## [A Primer on Testing with TestingBot](https://testingbot.com/resources/courses/testing-with-testingbot) Learn more on how to test with TestingBot - **Duration:** 10 minutes - **Difficulty:** Hard [Start course →](https://testingbot.com/resources/courses/testing-with-testingbot) [![Selenium Basics: TestNG](https://testingbot.com/assets/resources/courses/4-c9c7ea484cda943a8c695f9fa07a8257fe9047e66338206dcde2d2f0664275df.webp)](https://testingbot.com/resources/courses/selenium-testng "Selenium Basics: TestNG") Course ## [Selenium Basics: TestNG](https://testingbot.com/resources/courses/selenium-testng) Learn how to use TestNG in combination with Selenium to run automated, cross browser tests. - **Duration:** 30 minutes - **Difficulty:** Medium [Start course →](https://testingbot.com/resources/courses/selenium-testng) [![Preparing for Selenium Interview Questions](https://testingbot.com/assets/resources/courses/5-4f357dbaaf3a9aa3aba0aa09f90b952bea202004b1d7da021c419a7359dbca0c.webp)](https://testingbot.com/resources/courses/selenium-interview-questions "Preparing for Selenium Interview Questions") Course ## [Preparing for Selenium Interview Questions](https://testingbot.com/resources/courses/selenium-interview-questions) This course will help you prepare for answering any questions you might receive during a Selenium interview. - **Duration:** 100 minutes - **Difficulty:** Medium [Start course →](https://testingbot.com/resources/courses/selenium-interview-questions) --- URL: https://testingbot.com/resources/videos [Resources](https://testingbot.com/resources) / Videos # Videos Watch testing tutorials at your own pace. No videos match your filter. [![Selenium testing with Python Tutorial](https://testingbot.com/assets/videos/python-automated-bacffb33401f4772664d4d9c06455f2c0e4fabed0bfadf10770c0535a2bb6e76.png) ](https://testingbot.com/resources/videos/python-automated-testing-selenium "Selenium testing with Python Tutorial") Video Aug 19, 2021 ## [Selenium testing with Python Tutorial](https://testingbot.com/resources/videos/python-automated-testing-selenium) Want to use Python with Selenium? See this video to learn more. [Watch video →](https://testingbot.com/resources/videos/python-automated-testing-selenium) [![Automated Browser Testing with Ruby](https://testingbot.com/assets/videos/ruby-automated-a49902d15d265750dcd04b07a1fe142d8aac47976a9cae828e47835c01ce798f.png) ](https://testingbot.com/resources/videos/ruby-automated-testing-selenium "Automated Browser Testing with Ruby") Video Aug 19, 2021 ## [Automated Browser Testing with Ruby](https://testingbot.com/resources/videos/ruby-automated-testing-selenium) Video tutorial on how to run Ruby Automated Test on TestingBot. [Watch video →](https://testingbot.com/resources/videos/ruby-automated-testing-selenium) [![Automated testing with PHP and Selenium](https://testingbot.com/assets/videos/php-automated-29a925d6233fb94334c6c833b7e41d3e1ba05e04c5a01dc163a0fddac15d6889.png) ](https://testingbot.com/resources/videos/php-automated-testing-selenium "Automated testing with PHP and Selenium") Video Aug 19, 2021 ## [Automated testing with PHP and Selenium](https://testingbot.com/resources/videos/php-automated-testing-selenium) Learn how to use PHP with Selenium to do automated browser testing. [Watch video →](https://testingbot.com/resources/videos/php-automated-testing-selenium) --- URL: https://testingbot.com/product-updates # Product Updates Stay up-to-date with TestingBot's latest product updates. [RSS](https://testingbot.com/product-updates.atom "Atom feed") All iOS Android Browsers Framework No updates match your filter. ## October 2026 2026-10-01 iOS ### Test on the iPhone Duo simulator before it ships ![Test on the iPhone Duo simulator before it ships](https://testingbot.com/assets/updates/2026/iphone-duo-0f6c57818cc82048b2e4a44c6f8b84370248d050129148c1b41543baa2df0a99.png) The iPhone Duo simulator on iOS 27.1 is now available on TestingBot for live and automated testing. Apple has not released the device yet, so you can check how your site or app behaves on a foldable iPhone before your users get one. In a [live test](https://testingbot.com/members/manual/new) you see both screens: the cover screen when the device is folded and the large inner screen when it is open. Use the Fold and Unfold button in the side panel to switch, and check that your layout adapts when the viewport changes mid-session. For automated tests, use `platformName: "iOS"`, `deviceName: "iPhone Duo"` and `version: "27.1"`. ## August 2026 2026-08-11 ### Vibium browser automation TestingBot now supports [Vibium](https://testingbot.com/support/web-automate/vibium), the open-source browser automation tool built for AI agents. Point the Vibium CLI or its MCP server at our grid and your agent drives a real cloud browser over WebDriver BiDi: no extra configuration, and every session comes back with the usual video and logs. Client libraries for JavaScript, Python and Java are covered in the [documentation](https://testingbot.com/support/web-automate/vibium). 2026-08-05 Android ### Samsung Galaxy Tab S11 in the real device cloud The Samsung Galaxy Tab S11 is now available as a physical device for [live and automated testing](https://testingbot.com/real-device-testing). Test your app or website on a real Android tablet, next to the phones already in the cloud, and catch the layout issues an emulator will not show you. ## July 2026 2026-07-16 ### Notifications in the dashboard ![Notifications in the dashboard](https://testingbot.com/assets/updates/2026/notifications-drawer-e2a05e02f63813ea1b8d968c39d2e652ef039903e6c3396d2a04cd3bbc10b286.png) The bell in the top bar now opens an inbox: @mentions on test comments, alerts about your account, and this Product Updates feed under "What's new". 2026-07-16 Browsers ### Chrome 150 and Edge 150 Chrome 150 and Microsoft Edge 150 are now available for automated, live and visual testing. 2026-07-07 ### GitHub pull request checks ![GitHub pull request checks](https://testingbot.com/assets/support/github-pr-checks/pr-check-1e5054cfb6a7fc84eed29a1b7480ca8d1829d71086bf641d8fceb5065036298f.png) Install our [GitHub App](https://testingbot.com/support/integrations/ci-cd/github-pr-checks) and every pull request gets a TestingBot check: pass or fail, the number of tests, and a link to the results. Failures are commented on the pull request with the failing test and its video. 2026-07-06 ### DevTools while you test live The Web Inspector in live mobile web sessions has been rebuilt. Dock it at the bottom or on the left, drag to resize, or pop it out into its own window. The device now reserves its own space instead of being covered, and rescales as you drag. Console, network and elements stay open while you keep interacting with the page. 2026-07-06 iOS ### Faster live testing on iOS simulators Live app testing on iOS simulators now streams over WebRTC. Sessions start faster, the video is sharper, taps and swipes land with far less delay, and the session renders inside a real device frame. Since 14 July this is the default for every iOS 18 and newer simulator. 2026-07-05 ### Gradle plugin for Espresso Our new [Gradle plugin](https://testingbot.com/support/integrations/ci-cd/gradle) uploads your app and test APK and starts an Espresso run as part of your build. One task, instead of a curl script. 2026-07-04 ### Insights API The new Insights API exposes the numbers behind your dashboard: pass rate, duration trends, flaky tests and error breakdowns, per build or over time. Wire it into your own reporting or use it as a gate in CI. See the [API reference](https://testingbot.com/support/api). ## June 2026 2026-06-30 ### Firebase App Distribution and Fastlane ![Firebase App Distribution and Fastlane](https://testingbot.com/assets/support/integrations/firebase/firebase2-a014e7d3cbe0f7c2700784005230e0e1378f2558bee358d87694789df8a96c0f.png) Two new ways to get your app onto our device cloud. Connect [Firebase App Distribution](https://testingbot.com/support/integrations/firebase-app-distribution) and pull a build straight from your Firebase project, or upload from your existing lane with the [Fastlane plugin](https://testingbot.com/support/integrations/fastlane). - [1](https://testingbot.com/product-updates) - [2](https://testingbot.com/product-updates?page=2) - [3](https://testingbot.com/product-updates?page=3) - [4](https://testingbot.com/product-updates?page=4) - [5](https://testingbot.com/product-updates?page=5) - [6](https://testingbot.com/product-updates?page=6) - [7](https://testingbot.com/product-updates?page=7) - [8](https://testingbot.com/product-updates?page=8) - [9](https://testingbot.com/product-updates?page=9) - [10](https://testingbot.com/product-updates?page=10) - [11](https://testingbot.com/product-updates?page=11) - [12](https://testingbot.com/product-updates?page=12) - [13](https://testingbot.com/product-updates?page=13) - [14](https://testingbot.com/product-updates?page=14) - [15](https://testingbot.com/product-updates?page=15) - [16](https://testingbot.com/product-updates?page=16) - [17](https://testingbot.com/product-updates?page=17) - [18](https://testingbot.com/product-updates?page=18) - [19](https://testingbot.com/product-updates?page=19) - [20](https://testingbot.com/product-updates?page=20) - [21](https://testingbot.com/product-updates?page=21) - [22](https://testingbot.com/product-updates?page=22) - [23](https://testingbot.com/product-updates?page=23) - [Next page](https://testingbot.com/product-updates?page=2) Link copied --- URL: https://testingbot.com/blog # TestingBot Blog TestingBot provides a cloud of browsers and devices, ready to run your Automated, Live and Visual tests. [![Announcing AI Test Insights](https://testingbot.com/assets/blog/2026/ai-insights.jpg)](https://testingbot.com/blog/ai-test-insights "Announcing AI Test Insights") ## [Announcing AI Test Insights](https://testingbot.com/blog/ai-test-insights) 2026-06-04 Use AI Insights to quickly discover why an automated test failed. [Read article →](https://testingbot.com/blog/ai-test-insights) [![Selenium vs Playwright 2026](https://testingbot.com/assets/blog/2026/selenium-vs-playwright-2026.jpg)](https://testingbot.com/blog/selenium-vs-playwright "Selenium vs Playwright 2026") ## [Selenium vs Playwright 2026](https://testingbot.com/blog/selenium-vs-playwright) 2026-05-21 Introduction: Why the Selenium vs Playwright Comparison Matters in 2026 The comparison of Selenium against Playwright has far transcended discussi... [Read article →](https://testingbot.com/blog/selenium-vs-playwright) [![Maestro Physical Device Testing](https://testingbot.com/assets/blog/2026/maestro-physical.jpg)](https://testingbot.com/blog/maestro-physical-device-testing "Maestro Physical Device Testing") ## [Maestro Physical Device Testing](https://testingbot.com/blog/maestro-physical-device-testing) 2026-01-19 When we first introduced Maestro Cloud Testing on TestingBot, we focused on providing a reliable service for running Maestro cloud tests on em... [Read article →](https://testingbot.com/blog/maestro-physical-device-testing) [![tvOS Physical Device Testing](https://testingbot.com/assets/blog/2025/appletv.jpg)](https://testingbot.com/blog/tvos-automated-testing "tvOS Physical Device Testing") ## [tvOS Physical Device Testing](https://testingbot.com/blog/tvos-automated-testing) 2025-10-15 TestingBot is excited to announce the availability of its first Smart-TV testing device: Apple TV 4K. You can now run automated app test... [Read article →](https://testingbot.com/blog/tvos-automated-testing) [![MacOS Tahoe Cloud Testing](https://testingbot.com/assets/blog/2025/tahoe.jpg)](https://testingbot.com/blog/macos-tahoe-beta-testing "MacOS Tahoe Cloud Testing") ## [MacOS Tahoe Cloud Testing](https://testingbot.com/blog/macos-tahoe-beta-testing) 2025-06-10 We're excited to announce that TestingBot now offers early access to Apple's upcoming macOS release: macOS 26, also known as macOS Tahoe. Th... [Read article →](https://testingbot.com/blog/macos-tahoe-beta-testing) [![ChromeOS Cloud Testing](https://testingbot.com/assets/blog/2025/chromeos.jpg)](https://testingbot.com/blog/chromeos-testing "ChromeOS Cloud Testing") ## [ChromeOS Cloud Testing](https://testingbot.com/blog/chromeos-testing) 2025-06-10 Today we are announcing that TestingBot now supports ChromeOS for both manual and automated testing. You can now run your tests on ChromeOS ma... [Read article →](https://testingbot.com/blog/chromeos-testing) [![Accessibility testing with TestingBot](https://testingbot.com/assets/blog/2025/accessibility_testing.jpg)](https://testingbot.com/blog/accessibility-web-testing "Accessibility testing with TestingBot") ## [Accessibility testing with TestingBot](https://testingbot.com/blog/accessibility-web-testing) 2025-05-26 We're happy to announce a new addition to our automated testing toolkit: Accessibility Testing. With just a few clicks you can now scan your web... [Read article →](https://testingbot.com/blog/accessibility-web-testing) [![Monitor your Shopify Store](https://testingbot.com/assets/blog/2025/shopify-app.webp)](https://testingbot.com/blog/shopify-store-monitoring "Monitor your Shopify Store") ## [Monitor your Shopify Store](https://testingbot.com/blog/shopify-store-monitoring) 2025-02-27 Protect your Shopify Store from downtime: automated monitoring of Shopify Stores with real browsers. [Read article →](https://testingbot.com/blog/shopify-store-monitoring) [![Generative AI test agent](https://testingbot.com/assets/blog/2025/testingbot-ai.webp)](https://testingbot.com/blog/testingbot-ai-agent "Generative AI test agent") ## [Generative AI test agent](https://testingbot.com/blog/testingbot-ai-agent) 2025-02-18 At TestingBot, we're always looking for ways to make test automation faster, easier, and more efficient. Today, we're excited to introduce... [Read article →](https://testingbot.com/blog/testingbot-ai-agent) [![Automated Testing with TestingBot during the Holiday Season](https://testingbot.com/assets/blog/2024/holidayseason.webp)](https://testingbot.com/blog/testing-holiday-season "Automated Testing with TestingBot during the Holiday Season") ## [Automated Testing with TestingBot during the Holiday Season](https://testingbot.com/blog/testing-holiday-season) 2024-11-19 As the holiday season approaches, businesses worldwide gear up for the busiest shopping days of the year: Black Friday, Cyber Monday, Christmas and... [Read article →](https://testingbot.com/blog/testing-holiday-season) [![Selenium 20th birthday](https://testingbot.com/assets/blog/2024/selenium20.webp)](https://testingbot.com/blog/selenium-20-years "Selenium 20th birthday") ## [Selenium 20th birthday](https://testingbot.com/blog/selenium-20-years) 2024-10-29 2024 marks the 20th anniversary of Selenium, the pioneering tool in web browser automation. What started as a side project by ThoughtWorks engineer... [Read article →](https://testingbot.com/blog/selenium-20-years) - [1](https://testingbot.com/blog) - [2](https://testingbot.com/blog?page=2) - [3](https://testingbot.com/blog?page=3) - [4](https://testingbot.com/blog?page=4) - [5](https://testingbot.com/blog?page=5) - [6](https://testingbot.com/blog?page=6) - [7](https://testingbot.com/blog?page=7) - [8](https://testingbot.com/blog?page=8) - [9](https://testingbot.com/blog?page=9) - [10](https://testingbot.com/blog?page=10) - [11](https://testingbot.com/blog?page=11) - [12](https://testingbot.com/blog?page=12) - [13](https://testingbot.com/blog?page=13) - [Next page](https://testingbot.com/blog?page=2) --- URL: https://testingbot.com/free-online-tools 100% Free Tools # Free Online Developer Tools Made Simple A curated collection of free online tools for developers, testers and designers. Format code, generate test data and convert between formats, all from within your browser. Every tool runs in your browser, needs no account and stores nothing. We built them for our own QA work, so most are shaped around test data and debugging: generating throwaway identities and addresses to fill forms with Selenium or Playwright, hashing and encoding values, inspecting IPs and headers, and checking colour contrast before an accessibility run. 30+ Free Tools Instant Results Private No Data Stored ## Code Formatter - ### [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) Prettify JSON code online with this free tool. Try it free - ### [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) Escape HTML code online with this free tool. Try it free - ### [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) Prettify your XML code online with this free tool. Try it free - ### [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) Remove whitespace (tabs, spaces and newlines) from any text. Try it free - ### [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) Convert multi-line text or code into a single line without any breaks. Try it free - ### [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) Prettify, format and beautify CSS code or minify it to reduce file size. Try it free ## Page Speed - ### [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) Speed up your website by preloading the critical CSS on your pages. Try it free ## Accessibility - ### [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) Check WCAG color contrast compliance for text and backgrounds. Try it free ## Network Tools - ### [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) Find out your public IP address, location, ISP, and other network information instantly. Try it free - ### [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) Get your current GPS coordinates from your browser with address lookup and map view. Try it free - ### [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) Test Cross-Origin Resource Sharing (CORS) configuration of any endpoint. Try it free - ### [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) Calculate network address, broadcast, host range, wildcard mask and CIDR notation. Try it free ## Time & Date Tools - ### [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) Convert Unix timestamps to human-readable dates and get the current Unix timestamp. Try it free - ### [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) Generate and test cron expressions with human-readable descriptions and next run times. Try it free ## Converters - ### [Puppeteer to Playwright](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) Convert Puppeteer code into Playwright code. Try it free - ### [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) Convert pixel values to REM units for responsive designs. Try it free ## Data Generators - ### [Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) Generate random avatars for your projects and prototypes. Try it free - ### [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) Generate placeholder images with custom dimensions for your designs. Try it free - ### [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) Generate fake credit card numbers online for testing. Try it free - ### [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) Generate a random number between any two numbers. Try it free - ### [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) Decode and parse JWT tokens to inspect their contents. Try it free - ### [Address Generator](https://testingbot.com/free-online-tools/random-address-generator) Generate a random postal address for testing. Try it free - ### [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) Generate details for a fake person for testing purposes. Try it free - ### [Hash Text](https://testingbot.com/free-online-tools/hash-text) Hash text using MD5, SHA1, SHA256, SHA512, and more. Try it free - ### [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) Generate random placeholder text for your designs. Try it free - ### [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) Generate a random IPv4 address for testing. Try it free - ### [GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) Generate random and unique GUIDs. Try it free - ### [Character Count](https://testingbot.com/free-online-tools/character-count) Count the number of characters in any piece of text. Try it free - ### [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) Convert ASCII text to its binary notation. Try it free - ### [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) Generate random colors in HEX notation. Try it free - ### [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) Generate OG and social media meta tags for your website. Try it free - ### [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) Generate and download QR codes with customizable options. Try it free - ### [Device Information](https://testingbot.com/free-online-tools/device-information) Get detailed info about your device, browser, and screen. Try it free - ### [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) Parse user agent strings to detect browser, OS, and device. Try it free - ### [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) Extract all images from any public website. Try it free - ### [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) Generate comprehensive test plans with scope, schedule and risks. Try it free ## Calculators - ### [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) Estimate total testing hours based on project parameters, complexity, and automation coverage. Try it free - ### [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) Calculate quality score, requirements coverage, pass rate and defect density. Try it free ## Testing - ### [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) Test selectors against HTML with snippets for Selenium, Cypress and Playwright. Try it free --- URL: https://testingbot.com/free-online-tools/free-avatar-generator [Tools](https://testingbot.com/free-online-tools)Data Generators # Avatar Generator Generate realistic placeholder photos Quickly generate realistic avatar images using our free avatar API. Perfect for developers and designers who need dummy user photos for testing, mockups or placeholder content. These randomly generated avatars of people are ideal for use as sample profile pictures in web and mobile apps. ## Random Avatar Generator ## Random Avatar https://testingbot.com/free-online-tools/random-avatar/{size} Example [See](https://testingbot.com/free-online-tools/random-avatar/300) https://testingbot.com/free-online-tools/random-avatar/300 The maximum allowed size is 1000 ## Direct Image ID https://testingbot.com/free-online-tools/random-avatar/200?img=3 ## Unique ID https://testingbot.com/free-online-tools/random-avatar/200?u=fake@testingbot.com https://testingbot.com/free-online-tools/random-avatar/200?u=u837kfei139kn Add an identifier and you will always get the same image. ## What is a Random Avatar Generator? This free tool generates realistic avatar photos that can be used as placeholder images in your web or mobile projects. It's ideal for developers and designers who need dummy user photos for testing or mockups. ## How can I use the avatars in my project? You can use the avatars as user profile pictures during development or testing. Simply request the API with a desired image size or unique identifier to get a usable image URL. They're great for UI placeholders in apps, dashboards, signup forms or even in unit tests and functional tests. ## Can I generate the same avatar for each user? Yes. By passing a unique identifier such as a username, email, or user ID to the API, the generator will always return the same avatar for that ID. This makes it deterministic and ideal for consistent placeholder images across sessions. You can use the `u=` parameter in the URL to specify a unique identifier. For example, `https://testingbot.com/free-online-tools/random-avatar/128?u=12345` will always return the same avatar for the ID 12345, with an avatar dimension of 128x128 pixels. ## Is it free to use the avatar API? Yes, this avatar generator is completely free to use with no registration required. You can request as many avatar images as needed for your development or testing environment. ## What sizes are supported by the API? You can request avatar images in any square dimension by specifying the size in the URL, such as `https://testingbot.com/free-online-tools/random-avatar/128` or `https://testingbot.com/free-online-tools/random-avatar/512`. The image will be returned at that width and height. The maximum allowed dimension is 1000 pixels. If you request a size larger than that, the service will return an image of 1000x1000 pixels. ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/placeholder-image-generator [Tools](https://testingbot.com/free-online-tools)Data Generators # Placeholder Images Generate custom size placeholders Placeholder images are used by designers to visualize a design before the final image is obtained. **Generate Placeholder Image** Width (pixels) Height (pixels) Maximum dimensions: 2000x2000 pixels ## Image Preview ![Generated placeholder](https://testingbot.com) Data URL (Base64) Copy This data URL can be used directly in HTML img tags or CSS background-image properties. ## What is a Placeholder Image Generator? A Placeholder Image Generator is a tool that creates temporary images with specified dimensions for use in design mockups, development, and prototyping. These images typically display the dimensions (like "400x300") prominently and serve as visual placeholders until final images are ready. The generator creates base64-encoded data URLs that can be used directly in HTML, CSS, or any application that accepts image data without requiring external file storage. ## Why are Placeholder images used? Placeholder images serve several important purposes in design and development: - **Design Visualization:** Help designers and stakeholders visualize layouts before final content is available - **Development Testing:** Allow developers to test responsive designs and image handling without waiting for final assets - **Content Planning:** Provide visual representation of space allocation for future images in websites and applications - **Client Presentations:** Enable showing design concepts to clients without placeholder content distracting from the overall design - **Performance Testing:** Help test page load times and layout behavior with images of specific dimensions - **Template Creation:** Essential for creating reusable design templates and themes - **Prototyping:** Speed up the prototyping process by providing instant placeholder content ## How does the Placeholder Image Generator work? The Placeholder Image Generator works using HTML5 Canvas technology: 1. **Input Dimensions:** You specify the desired width and height for your placeholder image (up to 2000x2000 pixels) 2. **Canvas Creation:** The tool creates an HTML5 canvas element with your specified dimensions 3. **Image Rendering:** It draws a gray background and renders the dimensions text (e.g., "400x400") in the center using canvas drawing APIs 4. **Base64 Encoding:** The canvas content is converted to a base64-encoded PNG data URL using the toDataURL() method 5. **Output Generation:** The resulting data URL can be copied and used directly in your code without requiring external image files 6. **Preview Display:** The tool also shows a visual preview of the generated placeholder image for verification This process happens entirely in your browser, ensuring privacy and instant results without server uploads. ## Why do designers and developers use image placeholders? Designers and developers use image placeholders for several practical reasons: - **Workflow Efficiency:** Continue working on layouts and functionality without waiting for final images from clients or photographers - **Consistent Testing:** Test designs with images of exact dimensions needed, ensuring proper layout behavior - **Client Communication:** Show design concepts clearly without "Lorem Ipsum" style distractions - **Responsive Design:** Test how designs adapt across different screen sizes with consistently sized placeholder content - **Performance Optimization:** Test page loading and rendering performance with known image dimensions - **Template Development:** Create reusable themes and templates with properly sized placeholder areas - **Quality Control:** Ensure image containers and styling work correctly before final content integration - **Documentation:** Provide clear examples in style guides and documentation showing required image dimensions - **Agile Development:** Maintain development momentum in agile workflows where content may arrive iteratively ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/random-address-generator [Tools](https://testingbot.com/free-online-tools)Data Generators # Address Generator Generate realistic test addresses Use this free tool to generate random addresses for a specific country. You can use this in your automated or manual tests, for example to fill in a form. ## Random Address Generator Country Germany Australia Canada United States Argentina Spain Mexico France Indonesia Italy Belgium Netherlands Poland Portugal Generated Address ## Random Address Generator API Use our API to generate realistic random addresses for various countries. These addresses are useful for testing forms, filling mock user data, or populating QA environments with location-based content. GET `https://api.testingbot.com/v1/free-tools/random-address` Optional query parameter: - `?country=fr` — generate address using country (e.g., `us`, `au`, `fr`, `pl`, etc.) Response: ``` { "address": "2436 Main Street, Springfield, IL 62704, United States" } ``` ### Supported countries - `de` — Germany - `us` — United States - `ca` — Canada - `es` — Spain - `fr` — France - `id` — Indonesia - `it` — Italy - `nl` — Netherlands - `pl` — Poland - `pt` — Portugal This endpoint returns a randomly generated address in a format appropriate for the selected country. It is suitable for form-filling simulations, address validation testing or automated UI flows. ## What is a Random Address Generator? A Random Address Generator will generate fake, randomly created postal addresses. These are not real addresses, they do not exist, which means they cannot be used in production applications. Instead, these generated addresses are used for testing mobile applications or websites, which require the input of a valid looking postal address. ## Why do I need an Address Generator? You can use this generator to create valid looking, random postal addresses. Our address generator allows you to specify the country, after which the tool will generate a new (but fake) address. During your manual or automated tests, you can use this to generate addresses which can be used for your tests. For example, with a form that requires a postal address. ## How does this random address generator work? We use an utility called Faker. This program will generate fake data, that looks like it's real data. Under the hood, Faker will generate data such as random addresses. You can use this yourself with NodeJS and Ruby as well. ## How do I generate a fake address with Faker and Ruby? Once you've installed the Faker gem, you can simply call one function to generate an address. Please see the example below. ```ruby Faker::Address.full_address ``` ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/critical-css-generator [Tools](https://testingbot.com/free-online-tools)Page Speed # Critical CSS Generator Boost your page load speed instantly Critical CSS is the CSS required to render the above-the-fold content. By preloading this CSS before the larger CSS file, your webpage will load faster. Improve your performance score and Core Web Vitals like First Contentful Paint (FCP). This in turn improves customer satisfaction and SEO scores for your website. Enter URL: Wait time (ms): In case you need to wait longer for Javascript or animations Viewport dimensions: 800x600 1024x768 1280x800 1280x960 1280x1024 1440x900 1680x1050 1600x1200 1920x1200 1920x1080 2560x1440 ![Generating CSS](https://testingbot.com/assets/loading-0406eae1b17f4903931d9817122fdfbe585651d8fb63e564cf9def4c609fb9f7.gif) Output: /\* Critical CSS will appear here \*/ This Critical CSS generator reduces render-blocking resources, minimizes bandwidth usage, and ensures faster perceived load times. The result is quicker page rendering and improved performance, vital for retaining visitors and boosting conversions. The generator is simple to use. You can use it for each of the webpages that you would like to optimize. Once added, your Pagespeed Insight metrics will improve drastically. ## Why should I use a Critical CSS Generator? CSS stylesheets block the rendering of pages. Pages might remain blank until the browser completes requesting, downloading, and parsing all stylesheets. To boost rendering of pages, minimizing the CSS load is crucial. Manually identifying critical CSS can be laborious. This generator streamlines the process by automating the identification of essential CSS. From an SEO perspective, quicker page loading times are crucial for search engine rankings. Google's algorithm prioritizes fast-loading websites, considering them more user-friendly. By optimizing Critical CSS, you create a streamlined browsing experience, signaling search engines that your site is responsive and valuable. Improved pagespeed contributes to lower bounce rates, longer user sessions, and ultimately, a positive impact on your SEO efforts. ## How do I add the Critical CSS generated to my webpage? Once you've generated the critical CSS for your page, you can add it to our webpage. See the example below on how to do this. ```html
Hello, world!
``` In the above example, you would put your critical CSS in the head section of the DOM. The other css files will appear in the body, which means they will be loaded later in the webpage lifecycle. By loading the critical CSS in the head of the page, all DOM elements located before the large css files will use the critical CSS at the start of the webpage rendering process. ## Does this work for mobile websites as well? If you have a responsive website, generating the critical CSS with our tool will also work on mobile devices. You will notice improved pagespeed scores on mobile devices. ## Does this tool remove unused CSS? It will not remove unused CSS. This tool will check your webpage and see what CSS is required to display the above the fold content of your page. This CSS can be added to the page before the other (larger) CSS file, resulting in faster rendering of your webpages. ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter [Tools](https://testingbot.com/free-online-tools)Converters # Puppeteer to Playwright Convert test scripts automatically This free tool will automatically convert any Puppeteer script or test to a Playwright script. Enter Puppeteer code: Converted Playwright Code: This online tool is built for developers and QA. It will replace existing Puppeteer syntax with Playwright syntax. You can quickly convert your existing Puppeteer scripts to Playwright, without having to manually do the conversion yourself. ## Why should I migrate from Puppeteer scripts to Playwright scripts? There are multiple reasons why you would want to start using Playwright. One reason is that Playwright runs on non-chromium based browsers, such as Firefox. Another reason might be that you want to use a library or specific Playwright extension. While both are very similar in the features they offer and even the syntax, there are some subtle differences. Puppeteer uses the CDP protocol to instrument browsers, whereas Playwright uses its own, custom protocol. ## How can I manually convert Puppeteer code to Playwright code? ### Changing the required package First, you need to change the require of the Puppeteer package to Playwright. You'll need to replace the code below. ```javascript const puppeteer = require('puppeteer'); (async () => { const browser = await puppeteer.launch(); const page = await browser.newPage(); ``` With a new require, requiring the Playwright package. ```javascript const { chromium } = require('playwright'); (async () => { const browser = await chromium.launch(); const page = await browser.newPage(); ``` ### Cookies Puppeteer handles cookies at the page level, whereas with Playwright you need to manipulate cookies at the `BrowserContext` level. This means that these Puppeteer cookie functions: - `page.cookies([...urls])` - `page.deleteCookie(...cookies)` - `page.setCookie(...cookies)` will become these functions with Playwright: - `browserContext.cookies([urls])` - `browserContext.clearCookies()` - `browserContext.addCookies(cookies)` ### Viewport Puppeteer's `page.setViewport` becomes `page.setViewportSize` in Playwright. ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/json-prettify [Tools](https://testingbot.com/free-online-tools)Code Formatter # JSON Prettifier Format and beautify JSON instantly To get started, please paste the JSON string in the first textarea. The second textarea will automatically generate the Prettified JSON. ## Enter Value ## Output JSON (JavaScript Object Notation) is a lightweight data-interchange format that is easy for humans to read and write and easy for machines to parse and generate. It is often used for transmitting data between a server and a web application as an alternative to XML. JSON is based on a subset of the JavaScript programming language and consists of key-value pairs. ## What is Prettify JSON? Prettify JSON is a term used to describe the process of formatting a JSON string in a more human-readable and well-organized way. JSON strings can sometimes be long, compact, and difficult to read, especially when they contain nested objects or arrays. When you prettify JSON, you format the JSON string with proper indentation, line breaks, and whitespace, making it easier for humans to understand and navigate. This formatting does not affect the data itself; it only changes the presentation of the JSON string for better readability. ## What is JSON pretty format? JSON pretty format refers to the well-structured, human-readable representation of a JSON string with proper indentation, line breaks, and whitespace. When a JSON string is formatted in a pretty format, it becomes easier for humans to read and understand the data it contains, especially when dealing with complex JSON objects. The pretty format does not alter the data itself; it only changes the presentation of the JSON string for better readability. The goal is to make the JSON string more visually appealing and organized, which can be particularly helpful when working with large or nested JSON structures. ## How to prettify JSON in JavaScript? To prettify JSON in JavaScript, you can use the built-in `JSON.stringify()` method with the optional space parameter. The space parameter specifies the indentation applied to the JSON string, allowing you to generate a human-readable, pretty format. Here's how you can do it: ```javascript const prettyJson = JSON.stringify(jsonData, null, 2); ``` ## How to resolve JSON parse error? Ensure that the JSON data is in a valid format with proper syntax. JSON data should have matching opening and closing brackets/braces, and all property names and string values should be enclosed in double quotes. Remove any trailing commas at the end of arrays or objects. Trailing commas are not allowed in JSON and can cause a parse error. Ensure that all values in the JSON are valid. JSON supports strings, numbers, booleans, null, arrays, and objects. If there are any invalid values, such as functions or undefined, remove or convert them to valid JSON values. ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/html-escape [Tools](https://testingbot.com/free-online-tools)Code Formatter # HTML Escaper Escape special characters safely This generator escapes characters in HTML, allowing you to use HTML in other contexts such as in a `pre` tag. Characters such as `<` and `>` will be escaped. ## Enter Value ## Output ## What are the benefits of using HTML Escape Online? Escaping HTML characters is an essential practice in web development to ensure the security and integrity of web applications. When you escape HTML characters, you convert special characters into their respective HTML entities, preventing them from being interpreted as HTML code and avoiding potential security vulnerabilities. Here are the main benefits of escaping HTML characters: - **Preventing Cross-Site Scripting (XSS) Attacks**: XSS attacks occur when malicious code is injected into a web page and executed in the context of a user's browser. By escaping HTML characters, you prevent user input from being treated as code, effectively neutralizing XSS vulnerabilities. - **Avoiding Broken Layouts** : If unescaped HTML characters are inserted into a web page, they can disrupt the layout and structure of the page. Escaping characters prevents these rendering issues. - **Safe Rendering of User Input** : Escaping HTML characters allows developers to safely render user-generated content, such as comments or forum posts, without the risk of executing potentially harmful scripts. ## How to escape HTML in JavaScript? You can escape HTML in Javascript by replacing various characters from the input string, such as seen in the example below. ```javascript const escapeHtml = (unsafe) => { return unsafe.replaceAll('&', '&').replaceAll('<', '<').replaceAll('>', '>') .replaceAll('"', '"').replaceAll("'", '''); }; const unsafeString = 'Hello & welcome!'; const safeString = escapeHtml(unsafeString); console.log(safeString); // Output:
Hello & welcome!
``` In this example, the `escapeHtml` function replaces special characters with their corresponding HTML entities. ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/xml-prettify [Tools](https://testingbot.com/free-online-tools)Code Formatter # XML Prettifier Format and beautify XML instantly Paste the XML in the textarea below. TestingBot will automatically prettify the XML and output it in the second textarea. ## Enter Value ## Output ## What is XML Prettify Tool? TestingBot offers a free tool to clean up and indent XML code. TestingBot will make sure the indentation of all XML code is correct. ## What are the benefits of using XML Prettify Online? Using XML prettify, or formatting XML in a human-readable and well-organized manner, offers several benefits that make working with XML documents easier and more efficient: - **Improved Readability** : Prettified XML is easier to read and understand, especially when dealing with complex XML structures. Proper indentation and line breaks make the hierarchy and nesting of elements and attributes clear, enhancing readability. - **Debugging and Troubleshooting** : Prettified XML helps developers quickly identify issues and errors in the XML code. When an error occurs, it becomes easier to pinpoint the problematic section, allowing for faster debugging and troubleshooting. - **Version Control** : When using version control systems like Git, prettified XML allows for cleaner and more informative changes in version history. This helps developers understand the evolution of the XML document over time. ## How to prettify XML in JavaScript? You can prettify XML in Javascript by using `DOMParser()` and `XSLTProcessor`, see the example below: ```javascript const prettifyXml = function(sourceXml) { const xmlDoc = new DOMParser().parseFromString(sourceXml, 'application/xml'); const xsltDoc = new DOMParser().parseFromString([ // describes how we want to modify the XML - indent everything '', ' ', ' ', // change to just text() to strip space in text nodes ' ', ' ', ' ', ' ', ' ', ' ', '', ].join('\n'), 'application/xml'); var xsltProcessor = new XSLTProcessor(); xsltProcessor.importStylesheet(xsltDoc); var resultDoc = xsltProcessor.transformToDocument(xmlDoc); var resultXml = new XMLSerializer().serializeToString(resultDoc); return resultXml; } ``` ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/remove-all-whitespace [Tools](https://testingbot.com/free-online-tools)Code Formatter # Whitespace Remover Clean up text in seconds This tool will remove all the whitespace from the text you enter in the box below. Tabs, spaces, newlines will be removed. You will receive a spaceless string back. ## Remove Whitespace Tool Enter text: Welcome to the online whitespace, tab and newline remover tool. This web-based tool is built to remove all spaces from an input. The generator is simple to use. Simply enter the text (with spaces) in the textarea above, click the button and you will receive the same text back, but without any newlines, spaces or tabs. ## Why should I strip whitespaces from a piece of text? Automated tests often involve comparing actual program output or data with expected results. If your expected results contain extra whitespaces, tabs, or newlines, these inconsistencies can lead to test failures, even if the program's functionality is correct. Stripping whitespaces from expected results ensures that your tests are more precise and that they focus on functionality rather than formatting. ## How can I strip whitespaces using Javascript? You can use this oneliner Javascript code to remove all whitespaces, tabs and newlines from a string: ```javascript return input.replace(/\s/g, '') ``` ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/text-to-one-line [Tools](https://testingbot.com/free-online-tools)Code Formatter # Text to One Line Convert multi-line to single line This free tool allows you to turn multi-line text or code into a single line without any breaks. **Enter Multi-Line Text** Input Text Replace Line Break With a Space Remove Extra Whitespaces Output (Single Line) Copy ## What Is Text To One Line Tool? The Text to One Line tool is a free online utility that converts multi-line text or code into a single line format. It removes line breaks and newlines, optionally replacing them with spaces or simply concatenating the lines together. This tool is particularly useful for formatting code snippets, JSON data, or any text that needs to be converted to a single line for compatibility or storage purposes. ## What are the usecases for Text To One Line Tools? Text to One Line tools have several practical applications: - **Code Formatting:** Converting multi-line code snippets for inline usage or documentation - **JSON Processing:** Minifying JSON data for API calls or configuration files - **CSV Data:** Converting multi-line text fields for proper CSV formatting - **SQL Queries:** Formatting complex multi-line SQL statements for logging or embedding - **Configuration Files:** Converting multi-line configurations to single-line format for certain systems - **Testing:** Preparing test data that needs to be in single-line format for automated tests - **Data Migration:** Converting text data formats during database migrations ## How to use Text to One Line Tool? Using the Text to One Line tool is simple: 1. Paste or type your multi-line text into the input textarea 2. Choose your options: - **Replace Line Break With a Space:** Replaces line breaks with spaces instead of removing them entirely - **Remove Extra Whitespaces:** Removes multiple consecutive spaces and trims whitespace 3. Click the "Convert to One Line" button 4. The converted single-line text will appear in the output area 5. Use the "Copy" button to copy the result to your clipboard ## What are the Advantages of this Tool? The Text to One Line tool offers several advantages: - **Free to Use:** Completely free online tool with no registration required - **Instant Processing:** Real-time conversion with immediate results - **Flexible Options:** Choose between different formatting options for your specific needs - **Copy Functionality:** One-click copy to clipboard for easy use - **No File Upload:** Works directly with pasted text, ensuring privacy and security - **Cross-Platform:** Works on any device with a web browser - **Developer-Friendly:** Perfect for developers working with code, JSON, or configuration files - **Large Text Support:** Can handle substantial amounts of text efficiently - **Preserve Formatting Options:** Choose how to handle spaces and whitespace according to your needs ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/credit-card-number-generator [Tools](https://testingbot.com/free-online-tools)Data Generators # Credit Card Generator Create valid test card numbers Generate fake Credit Card numbers with this online generator tool. You can use these valid credit card numbers in your automated tests, to test payment flows and payment checkouts. **Credit Card Number** A credit card number generator is a software tool that employs coding techniques to simulate random credit card numbers. Developers and testers utilize this tool to evaluate the functionality of web pages without resorting to actual credit card information. These generators apply Luhn's algorithm to produce credit card numbers and associated details. These mock details are then employed as test data when crafting test scripts for web pages. Additionally, individuals can utilize this tool to generate fictitious credit card numbers for online forms, aiding in the testing of credit card processing systems. ## Use Our API to Generate Test Credit Card Numbers Developers can now integrate our fake credit card number generator directly into their applications or test suites via our public API. This is ideal for automated testing, QA environments and mock data generation. GET `https://api.testingbot.com/v1/free-tools/fake-credit-card` Response: ``` { "credit_card_number": "4929429829012934" } ``` The API returns a Luhn-valid fake credit card number in JSON format. You can call this endpoint programmatically to fetch a fresh test number each time. No authentication required. ## What is the mechanism behind a credit card number generator? - The **Major Identity Identifier** (MII) denotes the initial digit of the credit card number, signifying the category of the credit card provider. When using our credit card number generator tool, you can expect precise MII representation. - The **Issuer Identification Number** (IIN) represents the first six digits of a credit card number. This segment serves the purpose of identifying the institution responsible for issuing the credit cards. - **Luhn's Algorithm** stands as a validation formula essential for verifying credit card numbers. The TestingBot generator tool leverages Luhn's Algorithm in its backend operations, ensuring the provisioning of authentic credit card numbers, suitable for use in test data with your automated tests. ## Why use a Credit Card Number Generator? - Test **payment checkout pages** : Considering the sensitive nature of credit card data and the necessity for testers and developers to employ them frequently during the evaluation of purchase pages or processes, it becomes unsafe to include genuine credit card details in test data. With a credit card number generator, a fake (simulated) credit card numbersclosely resembling an authentic one, makes sure for valid testing while keeping everything secure. - Some payment providers supply their own dummy credit card numbers for testing/validation. With our tool, you have access to additional numbers for testing. Our generator is capable of generating thousands of various Visa/American Express and other credit cards. - Perform payment gateway testing by inputting the generated credit card numbers, either manually or during an automated test. ## Is it legal to use the TestingBot Credit Card Generator? Yes, this is definitely legal as these credit card numbers look like real numbers but cannot actually be used to perform purchases. Please use the generated numbers only in tests, against apps that allow you to perform this type of testing. ## Are the generated credit card numbers safe to use for sensitive transactions? These generated credit card numbers are primarily designed for testing purposes. We do not recommend using these during any sensitive tests/transactions. ## Can I make purchases using these credit card details? It is not possible to use these for real purchases, as the data that our generator outputs lacks information for a valid payment, such as: - A proper expiration date of the card - The cardholder's name - CVV numbers ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/random-number-generator [Tools](https://testingbot.com/free-online-tools)Data Generators # Random Number Generator Generate numbers in any range If you are looking to generate random numbers with a simple to use tool, then you've found it with this Free Number Generator. Simply fill in a minimum and maximum value, the generator will randomly pick a number between these two values. ## Random Number Generator Minimum Maximum Random Number ## What is a Random Number Generator? A random number generator (RNG) will generate unique, non-repeating numbers between two values that you specify. By default, it will output a random number between 0 and 100. By changing the minimum and maximum values, you can tweak the process. ## Why do I need a Random Number Generator? This generator might be useful when you need to test an input form where a number is required. To make sure you cover all test cases, you might want to make the number unique, to make sure it does not trigger any duplicate errors. Let's say you are testing a form that requires a house number. You might want to make sure that you are always including a random house number during the test. ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/jwt-parser [Tools](https://testingbot.com/free-online-tools)Data Generators # JWT Parser Decode and inspect JWT tokens This online JWT parser will display all the values embedded in the JWT token. This can be useful to investigate issues with JWT tokens used in web apps. Enter your JWT token: ## Header | Parameter | Value | | --- | --- | ## Payload | Claim | Value | | --- | --- | ## What is a JWT Parser? A JWT Parser is a tool that takes a JSON Web Token (JWT) as input and decodes it to reveal the contents of the header and payload sections. JWTs are encoded in Base64url format and consist of three parts: header, payload and signature. Parsing a JWT allows developers and testers to inspect the claims and metadata contained within the token without validating the signature. ## When should I use a JWT Parser? You should use a JWT Parser when you need to examine the contents of a JWT, such as during development, debugging or testing of authentication and authorization mechanisms in web applications and APIs. It helps in verifying that the correct claims are included, that they have the expected values and that the token is structured properly. This is particularly useful for developers and QA engineers performing automated tests on platforms like TestingBot, where ensuring correct token generation and handling across different browsers and devices is essential. ## How can a JWT Parser help with automated testing? A JWT Parser can assist in automated testing by allowing test scripts to programmatically inspect and validate the contents of JWTs generated by an application. This ensures that the tokens contain the correct claims and that sensitive information is not unintentionally exposed. ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/fake-person-generator [Tools](https://testingbot.com/free-online-tools)Data Generators # Fake Person Generator Create realistic test identities Use this free tool to generate fake person details, including a name, address and social security number. You can use this in your automated or manual tests, for example to fill in a form requesting personal details. ## Random Person Generator Country Germany Australia Canada United States Argentina Spain Mexico France Indonesia Italy Belgium Netherlands Poland Portugal ## What is a Random Person Generator? A Random Person Generator will generate a fake, randomly created person with their full details. These are not real personal details, they do not exist, which means they cannot be used in production applications for testing. These generated personal details are used for testing mobile applications or websites, which require the input of valid looking personal details, such as an address, birthday, gender, phone number and more. ## Why do I need an Person Generator? You can use this generator to create valid looking but random personal details. Our person generator allows you to specify the country, after which the tool will generate a new (but fake) person. You can use this generated information in your tests, for example to test a form that requires the input of a person's details. ## How does this random people generator work? We use an utility called Faker. This is a tool that will generate fake data, but looks like it's real data. Under the hood, Faker will generate data such as random personal details. You can use this tool yourself with NodeJS and Ruby as well. ## How do I generate a fake person with Faker and Ruby? Once you've installed the Faker gem, you can use various functions to generate the necessary information. Please see the example below. ```ruby name = Faker::Name.name address = Faker::Address.full_address ``` ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/hash-text [Tools](https://testingbot.com/free-online-tools)Data Generators # Hash Text Generate MD5, SHA and more This free online hash text generator will convert the text that you input to various hashes, with a specific Digest encoding. Start typing to see the various hashes, ranging from SHA1 to SHA256, MD5 and RIPEMD160. ## Hash Text Generator Your text to hash: Digest encoding Base64 (base 64) Binary (base 2) Hexadecimal (base 16) | Algorithm | Hash Output | | --- | --- | | MD5 | | | SHA1 | | | SHA256 | | | SHA224 | | | SHA512 | | | SHA384 | | | SHA3 | | | RIPEMD160 | | ## What is a Hash Text Generator? A Hash Text Generator is a tool that takes an input string (text) and computes a cryptographic hash of that string using a specified hash algorithm. The output is a fixed-size string of characters, which appears random and is typically used for data integrity verification, password storage or as a unique identifier for data. Common hash algorithms include MD5, SHA1, SHA256, SHA512, and others. The generated hash is deterministic, which means the same input will always produce the same hash output. ## Why do I need a Hash Text Generator? A Hash Text Generator is useful for various purposes: - **Data Integrity Verification** : By comparing hash values, you can verify that data has not been altered or corrupted during transmission or storage. - **Unique Identifiers** : Hashes can serve as unique identifiers for files or data blocks, useful in scenarios like deduplication or cache keys. - **Testing and Development** : Developers and QA engineers can use a Hash Text Generator to create test cases, verify hashing algorithms or ensure that applications handle hash values correctly. Hashes can be used to verify data integrity across different environments and browsers. ## How is a hash text generator built? A Hash Text Generator is typically built using cryptographic hash functions provided by programming languages or libraries. In JavaScript, for instance, you can use the Web Crypto API for modern browsers or external libraries like CryptoJS for broader compatibility. The basic steps to build a hash text generator include: 1. **Input Handling** : Capture the text input from the user interface. 2. **Algorithm Selection** : Allow the user to select which hash algorithm to use (e.g., MD5, SHA1, SHA256). 3. **Hash Computation** : Use the selected algorithm to compute the hash of the input text. 4. **Digest Encoding** : Convert the hash output into a human-readable format, such as hexadecimal, base64, or binary encoding. 5. **Display Output** : Present the computed hash to the user. ## What are common use cases for a hash text generator in automated testing? In automated testing, a hash text generator can be used to: - **Verify Data Consistency** : By hashing outputs from different environments or browsers, testers can ensure that the data remains consistent across platforms. - **Detect Changes** : Hashes can detect even minor changes in data output, which is helpful when verifying that applications behave identically across different browsers during cross-browser testing on platforms like TestingBot. - **Security Testing** : Testing the implementation of hashing functions in applications, ensuring that passwords and sensitive data are hashed correctly before storage or transmission. - **Performance Benchmarking** : Comparing the performance of hashing algorithms across different devices and browsers. ## How does a hash function work? A hash function takes input data of arbitrary size and produces a fixed-size string of characters, which is typically a digest that represents the input data uniquely. Cryptographic hash functions are designed to be one-way functions, meaning it is computationally infeasible to reverse the output back to the original input. They are also designed so that small changes in the input produce significantly different outputs, a property known as the avalanche effect. ## What are the differences between MD5, SHA1, SHA256 and other hash algorithms? The primary differences between these hash algorithms are their output size and security levels: - **MD5** : Produces a 128-bit hash value. It is fast but considered cryptographically broken due to vulnerabilities that allow for hash collisions. - **SHA1** : Produces a 160-bit hash value. It has been deprecated for use in security-critical applications because of known vulnerabilities. - **SHA256** : Part of the SHA-2 family, it produces a 256-bit hash value and is considered secure for most applications. - **SHA3** : The latest standard, designed to be secure against attacks that may affect SHA-2. - **RIPEMD160** : Produces a 160-bit hash value, similar to SHA1, but with a different design. ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/lorem-ipsum-generator [Tools](https://testingbot.com/free-online-tools)Data Generators # Lorem Ipsum Generator Create placeholder text instantly Welcome to the free online Lorem Ipsum Generator. This free tool generates random pieces of Lorem Ipsum text. Random Text ## What is a Lorem Ipsum Generator? A Lorem Ipsum generator is a tool which will generate random sentences or words. It is used when you need to type or enter text in an application or website, without it having to be real text. ## Why do I need a Lorem Ipsum Generator? You can use this generator to generate random pieces of text, which can be used during manual or automated testing. For example, when you need to fill in a form on a web-app, but you don't want to fill in real text (due to privacy reasons, or legal reasons). It is an easy way to provide a placeholder text, when the actual content of your webpage is not yet available. Instead of leaving the space blank, you can fill it up with random text, allowing designers and developers to make sure the layout looks correct. ## What does Lorem Ipsum mean? In both written and visual contexts, the expression "lorem ipsum" represents a placeholder text used to fill content gaps in documents. "Dolorem Ipsum", roughly translates to "pain itself" in Latin, serves as the linguistic origin of the "lorem ipsum" term. The first sentences of a Lorem generator are usually something like this: "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed do eiusmod tempor incididunt ut labore et dolore magna aliqua." Because it is written in Latin, it does not distract developers or designers while creating a page, since most people don't know how to read Latin and assume it's gibberish. ## How can I generate Random Lorem Ipsum text with Ruby? The Faker gem on Ruby allows you to generate random pieces of text. Please see the example below on how to do this. ```ruby require('faker') Faker::Lorem.paragraph(sentence_count: 20) ``` ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/random-ip-generator [Tools](https://testingbot.com/free-online-tools)Data Generators # Random IP Generator Generate valid IPv4 addresses Generate random IP addresses, to be used in your tests, for free with this online IP generator. These are valid IPs that can be used for testing your website or mobile app. ## Random IP Generator Generated IP Address ## What is a Random IP Generator? A random IP address generator will create valid looking, random IP addresses, which can be used in your tests (automated or manual). If your test, or the website/mobile app you are testing, requires a valid IP address, you can use this tool to create dummy IPs. ## Why do I need an IP Address Generator? The generator will generate a valid, public IP address. This IP address can be used in your automated or manual tests when required by the website or mobile app you are testing. ## What is the difference between an IPv4 and IPv6 address? The IPv4 address is the original format of an IP address and is still used by over 90% of all internet appliances. It consists of a 32-bit binary number, separated by dots. An IPv6 address is an 128-bit binary number, separated by colons. As it is 128 bit, the number of possibilities is exponentially bigger than the original IPv4 address. ## How do I generate an IP address in my test cases? This depends on the programming language you are using. An example on how to do this with Javascript or NodeJS can be found below. ```javascript const randomIp = Array(4).fill(0).map((_, i) => Math.floor(Math.random() * 255) + (i === 0 ? 1 : 0)).join('.'); ``` ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/random-guid-generator [Tools](https://testingbot.com/free-online-tools)Data Generators # GUID Generator Create unique identifiers This free online tool allows you to generate random GUID (Globally Unique Identifier) or UUIDs (Universally Unique Identifier). ## Random GUID Generator Number of GUIDs to generate Results ## What is a random GUID generator? You can easily generate Globally Unique Identifiers (GUIDs) with this Random GUID Generator. These uniquely identifying strings are generated locally using JavaScript within the online GUID generator, to make sure no data is transmitted over the internet. You can use GUID identifiers in programs or testcases. If you need to store an item or query a unique thing from a database, we recommend using GUIDs as these are unique strings that cannot be guessed. In contrast with auto-incremented identifiers, these identifiers are much more secure to use. ## Why use Random GUID Generator? Utilizing a Random GUID Generator offers several advantages and use cases: - **Global Uniqueness:** GUIDs (Globally Unique Identifiers) generated by a Random GUID Generator are designed to be globally unique. The randomization process significantly reduces the likelihood of duplication, ensuring that each generated identifier is highly distinctive. - **Security:** Random GUIDs enhance security by providing a level of unpredictability. The random nature of the generated identifiers makes it challenging for malicious entities to predict or manipulate them, contributing to improved security in various applications. - **Data privacy:** When using this online Random GUID Generator, the generation process loccurs locally in your browser. This means that the data used to create the GUIDs doesn't need to be transmitted over the internet, contributing to enhanced data privacy. - **Promo Code Generation:** Random GUIDs are often employed in e-commerce for generating unique promo codes. This ensures that each promotional code is distinct, reducing the risk of abuse or exploitation. You could for example use this as mock-data for a testcase that needs to test a coupon code in a shopping cart. ## How unique is a GUID? A GUID is designed to be extremely unique, but the degree of uniqueness is not absolute. GUIDs are 128-bit values and their uniqueness is primarily achieved through a combination of factors: - **Randomness:** Many modern GUIDs are generated using algorithms that incorporate random or pseudo-random components. This randomness contributes significantly to uniqueness, as the likelihood of two randomly generated GUIDs being identical is very low. - **Timestamps:** Some GUIDs include a timestamp component, which is a combination of the current time and the unique identifier of the computer or network card generating the GUID. This helps ensure uniqueness, especially when GUIDs are generated at different times, or generated on different machines. When high levels of uniqueness are required, additional measures, such as using GUIDs with timestamps or other unique elements, can be implemented. ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/character-count [Tools](https://testingbot.com/free-online-tools)Data Generators # Character Counter Count characters and words This free online tool allows you to count the number of characters in a piece of text. **Enter any piece of text** Text Character Count ## What is a Character Counter? A character counter is a tool that will calculate how many characters there are in a piece of text. It does this by going over all the individual characters and incrementing a counter. ## Why do I need a Character Counter? You might need to know the count of characters during an automated test, to make certain assertions when validating a specific flow. For example, you might expect a website or mobile app to return a piece of text. Using the character counter, you can make sure the text has the right number of characters. ## How can I count the characters of a string with Javascript? You can use the `.length` property of any string with Javascript. This property will return the amount of characters in the given string. ```javascript const myString = "this is a piece of text"; const characterCount = myString.length; ``` ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/ascii-to-binary [Tools](https://testingbot.com/free-online-tools)Data Generators # ASCII to Binary Convert text to binary code Use this free online generator to convert ASCII text to Binary Notation. ASCII Binary Output This online tool is built for developers and QA. It allows you to convert ASCII text to its binary representation. You can quickly convert any ASCII text to binary without manual calculation. ## Why should I need to convert ASCII to binary? Converting ASCII (American Standard Code for Information Interchange) to binary is necessary in certain contexts for various reasons: - **Data Representation:** Binary is the fundamental language of computers. Converting ASCII to binary allows computers to process and store textual data as a sequence of binary digits (ones and zeroes). - **Data Transmission:** When transmitting data over a network, binary is often used to represent characters, numbers, and other information. - **Encryption and Encoding:** In cryptographic operations and data encoding techniques, converting text to binary is a common step. Binary representations often enable various encryption algorithms and encoding methods to secure and transmit data more effectively. ## How do I convert ASCII to binary manually? You can manually convert ASCII to binary by using a conversion table or formula. For each character, you can find its decimal value. Once you have the decimal value, you can convert it to binary. ## How can I convert ASCII to binary with JavaScript? You can use the `charCodeAt` method to convert a character to its binary value: ```javascript const character = "a" const binary = character.charCodeAt(0).toString(2) ``` ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/random-color-generator [Tools](https://testingbot.com/free-online-tools)Data Generators # Random Color Generator Generate HEX color codes This free online tool allows you to generate random colors, which can be used in your apps or tests. ## Random Color Generator Color Value ## What is a Random Color Generator? A Random Color Generator is a software tool or algorithm that generates random colors. You can use this generator to create colors which can be used in your automated test cases, for example as an input in a specific form. The generator on this page will create unique HEX based colors, which you can copy/paste into your test scripts. ## How can I generate random colors in my test scripts? You can use this simple code below in Javascript which will generate unique HEX colors each time you call this method. ```javascript Math.floor(Math.random()*16777215).toString(16) ``` ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/color-contrast-checker [Tools](https://testingbot.com/free-online-tools)Accessibility # Color Contrast Checker Check WCAG accessibility compliance Enter foreground/background colors to get WCAG accessibility compliance info. Ensure your text is readable for all users. ## WCAG Color Contrast Checker Foreground Color (Text) Background Color Sample Text Preview The quick brown fox jumps over the lazy dog. This shows how your color combination looks in different text sizes. ### Contrast Ratio 21:1 #### Normal Text WCAG AA PASS WCAG AAA PASS #### Large Text WCAG AA PASS WCAG AAA PASS #### WCAG Requirements **Normal Text:** - • AA: 4.5:1 minimum - • AAA: 7:1 minimum **Large Text (18pt+ or 14pt+ bold):** - • AA: 3:1 minimum - • AAA: 4.5:1 minimum Swap Colors ## What is color contrast? Color contrast refers to the difference in light between font (or anything in the foreground) and its background. In web accessibility, color contrast is a measure of the difference in perceived brightness between two colors. This difference is expressed as a ratio ranging from 1:1 (no contrast, same color) to 21:1 (maximum contrast, black on white or vice versa). The Web Content Accessibility Guidelines (WCAG) define specific contrast ratios to ensure text is readable for users with visual impairments, including color blindness and low vision conditions. Proper color contrast ensures that content is perceivable by the widest possible audience. ## Why is color contrast important for accessibility? Color contrast is crucial for web accessibility for several important reasons: - **Visual Impairments:** Approximately 285 million people worldwide have visual impairments. Sufficient contrast helps users with low vision, cataracts, or age-related vision loss read content more easily. - **Color Blindness:** About 8% of men and 0.5% of women have some form of color blindness. Good contrast ratios ensure content remains readable regardless of color perception differences. - **Environmental Factors:** Users may view content on devices in bright sunlight, on low-quality screens, or with glare, all of which reduce perceived contrast. - **Cognitive Load:** Better contrast reduces eye strain and cognitive load for all users, making content easier to read and understand. - **Legal Compliance:** Many countries require websites to meet WCAG standards, making proper contrast a legal requirement for public and commercial websites. ## How can I ensure proper color contrast in my designs? Follow these best practices to ensure your designs meet color contrast requirements: - **Test Early and Often:** Use this color contrast checker tool during the design phase, not after implementation. Test all text and important visual elements against their backgrounds. - **Follow WCAG Guidelines:** Aim for WCAG AA compliance at minimum (4.5:1 for normal text, 3:1 for large text). Consider AAA compliance (7:1 and 4.5:1 respectively) for critical content. - **Don't Rely on Color Alone:** Use additional indicators like underlines for links, icons, or text labels to convey information. - **Test with Real Users:** Conduct usability testing with users who have visual impairments to validate your contrast choices. - **Consider Different States:** Check contrast for all interactive states (hover, focus, active, disabled) and ensure they all meet requirements. - **Use Design Systems:** Create a color palette with pre-tested combinations that meet contrast requirements to ensure consistency across your project. ## What are WCAG contrast ratio requirements? The Web Content Accessibility Guidelines (WCAG) 2.1 define specific contrast ratio requirements for different levels of compliance: ### Level AA (Minimum Compliance) - **Normal Text:** Minimum contrast ratio of 4.5:1 - **Large Text:** Minimum contrast ratio of 3:1 - **Graphics & UI Components:** Minimum contrast ratio of 3:1 ### Level AAA (Enhanced Compliance) - **Normal Text:** Minimum contrast ratio of 7:1 - **Large Text:** Minimum contrast ratio of 4.5:1 **Note:** Large text is defined as 18pt (24px) or larger for regular weight, or 14pt (18.5px) or larger for bold weight. Level AAA is not required for entire websites but is recommended for critical content. ## How is the contrast ratio calculated? The contrast ratio is calculated using the relative luminance of the two colors according to the WCAG formula: Contrast Ratio = (L1 + 0.05) / (L2 + 0.05) Where L1 is the relative luminance of the lighter color and L2 is the relative luminance of the darker color. The relative luminance is calculated by: 1. Converting RGB values to sRGB (values between 0 and 1) 2. Applying gamma correction to each channel 3. Calculating weighted sum: 0.2126 \* R + 0.7152 \* G + 0.0722 \* B This tool automatically performs these calculations for you, instantly showing whether your color combinations meet WCAG standards for both normal and large text sizes. ## What tools can help with color contrast testing? Several tools and techniques can help you test and ensure proper color contrast: - **This Color Contrast Checker:** Use our free tool above to test any color combination instantly against WCAG standards. - **Browser DevTools:** Chrome and Firefox DevTools include built-in contrast ratio checkers in their color pickers. - **Design Software Plugins:** Figma, Sketch, and Adobe XD have plugins like Stark or Able that check contrast ratios during design. - **Automated Testing:** Tools like axe DevTools, WAVE, or Lighthouse can scan entire pages for contrast issues. - **Manual Testing:** Use browser extensions like WCAG Color Contrast Checker or Colour Contrast Analyser (CCA) for on-page testing. - **Screen Readers:** Test with actual assistive technologies to understand the real user experience. Remember to test contrast in different contexts: on various devices, under different lighting conditions, and with different display settings to ensure accessibility for all users. ### All Tools #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/cors-tester [Tools](https://testingbot.com/free-online-tools)Network Tools # CORS Tester Test cross-origin configuration Test Cross-Origin Resource Sharing (CORS) configuration by sending a fetch request to any endpoint and checking if it passes or fails CORS validation. Endpoint URL HTTP Method GET POST PUT DELETE PATCH HEAD OPTIONS Custom Headers (Optional) One per line, format: Header-Name: value Request Body (Optional) For POST, PUT, PATCH requests Test CORS ### Response Headers | Header | Value | | --- | --- | ### Response ## What is CORS? CORS (Cross-Origin Resource Sharing) is a security feature implemented by web browsers that controls how web pages from one domain can access resources from another domain. It's a mechanism that uses HTTP headers to tell browsers whether to allow a web application running at one origin to access selected resources from a server at a different origin. By default, browsers enforce the Same-Origin Policy, which blocks requests between different origins. CORS provides a controlled way to relax this restriction, allowing servers to specify which origins are permitted to access their resources. ## How to Enable CORS in Different Programming Languages? ### Node.js (Express) ```javascript // Install cors package: npm install cors const cors = require('cors'); app.use(cors({ origin: 'https://example.com', credentials: true })); ``` ### Python (Flask) ```python # Install flask-cors: pip install flask-cors from flask_cors import CORS CORS(app, origins=['https://example.com']) ``` ### Ruby on Rails ```ruby # In config/initializers/cors.rb Rails.application.config.middleware.insert_before 0, Rack::Cors do allow do origins 'example.com' resource '*', headers: :any, methods: [:get, :post, :put, :patch, :delete, :options, :head] end end ``` ### PHP ```php header("Access-Control-Allow-Origin: https://example.com"); header("Access-Control-Allow-Methods: GET, POST, PUT, DELETE"); header("Access-Control-Allow-Headers: Content-Type"); ``` ### Java (Spring Boot) ```java @CrossOrigin(origins = "https://example.com") @RestController public class ApiController { // Your endpoints } ``` ## What are CORS Preflight Requests? A CORS preflight request is an OPTIONS request that browsers automatically send before certain cross-origin requests. This happens when the request uses methods other than GET, HEAD, or POST, or when it includes custom headers. The preflight request checks with the server whether the actual request is allowed. The server responds with headers indicating which origins, methods, and headers are permitted. Only if the preflight succeeds will the browser send the actual request. Understanding preflight requests is crucial for debugging CORS issues, as failures in the preflight phase will prevent your actual request from being sent. ## Common CORS Error Messages and Solutions Error: "No 'Access-Control-Allow-Origin' header" Solution: The server needs to include the Access-Control-Allow-Origin header in its response. Configure your server to add this header with appropriate origin values. Error: "CORS header 'Access-Control-Allow-Origin' missing" Solution: Similar to above, ensure your server is configured to send CORS headers. Check that your server-side CORS configuration is properly set up. Error: "Credentials flag is true but Access-Control-Allow-Credentials is not 'true'" Solution: When sending credentials (cookies, auth headers), the server must include `Access-Control-Allow-Credentials: true` header, and `Access-Control-Allow-Origin` must not be a wildcard (\*). ## CORS Headers Explained - **Access-Control-Allow-Origin:** Specifies which origins can access the resource. Use specific domains for security, not wildcard (\*) in production. - **Access-Control-Allow-Methods:** Lists the HTTP methods allowed when accessing the resource (GET, POST, PUT, DELETE, etc.). - **Access-Control-Allow-Headers:** Indicates which headers can be used during the actual request. - **Access-Control-Allow-Credentials:** Indicates whether the response can be shared when credentials flag is true. - **Access-Control-Max-Age:** Indicates how long the results of a preflight request can be cached. - **Access-Control-Expose-Headers:** Lists headers that the browser can access from the response. ## How to Test CORS Configuration? Testing CORS configuration is essential to ensure your API is accessible from authorized origins while maintaining security. Use this CORS tester tool to: - Verify that your API endpoints are properly configured for cross-origin requests - Test different HTTP methods (GET, POST, PUT, DELETE) to ensure they're allowed - Check if custom headers are accepted by your server - Validate preflight request handling for complex requests - Debug CORS errors by examining response headers - Test authentication scenarios with credentials Regular CORS testing helps prevent issues in production and ensures smooth integration with frontend applications hosted on different domains. ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/my-gps-location [Tools](https://testingbot.com/free-online-tools)Network Tools # My GPS Location Get your coordinates from browser Find your current GPS coordinates from your browser. Get your latitude, longitude and address details instantly. ## What are my coordinates? Click the button below to detect your current GPS location Detect My Location Detecting your location... Try Again ### My location coordinates are: Latitude Longitude Copy Coordinates Refresh ### What is my location? Looking up address... My location address is: City: - State: - Country: - Postal Code: - Timezone: - What3Words: - Could not retrieve address information ### GPS Details Accuracy: - Altitude: - Heading: - Speed: - Geohash: - Timestamp: - ### Where am I? My Location Map ## What is GPS location? GPS (Global Positioning System) location refers to the coordinates that pinpoint your exact position on Earth. It uses a constellation of satellites orbiting Earth to determine your location with high precision. Your GPS location consists of two main components: - **Latitude:** Your north-south position, measured in degrees from -90 (South Pole) to +90 (North Pole), with 0 at the equator - **Longitude:** Your east-west position, measured in degrees from -180 to +180, with 0 at the Prime Meridian (Greenwich, London) Modern smartphones and browsers can access GPS data to provide location-based services, navigation, weather information and more. ## GPS coordinates: how do they work? How accurate are they? GPS works by receiving signals from multiple satellites (at least 4) and calculating your position based on the time it takes for each signal to reach your device. This process is called **trilateration**. **Accuracy levels:** - **Standard GPS:** 3-5 meters accuracy under open sky conditions - **Assisted GPS (A-GPS):** Uses cell towers and Wi-Fi to improve accuracy and speed of location fix - **DGPS (Differential GPS):** Can achieve sub-meter accuracy using ground-based reference stations - **RTK GPS:** Real-Time Kinematic GPS can achieve centimeter-level accuracy **Factors affecting accuracy:** - Number of visible satellites - Atmospheric conditions (ionosphere interference) - Urban canyons (tall buildings blocking signals) - Indoor vs outdoor positioning - Device hardware quality ## How can I get GPS coordinates in Java, C#, Ruby, Python and NodeJS? Here's how to get GPS coordinates programmatically in different languages: **JavaScript (Browser):** ```javascript navigator.geolocation.getCurrentPosition( (position) => { const lat = position.coords.latitude; const lng = position.coords.longitude; console.log(`Latitude: ${lat}, Longitude: ${lng}`); }, (error) => console.error(error) ); ``` **Python:** ```python import geocoder g = geocoder.ip('me') print(f"Latitude: {g.lat}, Longitude: {g.lng}") ``` **Java (Android):** ```java LocationManager lm = (LocationManager) getSystemService(Context.LOCATION_SERVICE); Location location = lm.getLastKnownLocation(LocationManager.GPS_PROVIDER); double lat = location.getLatitude(); double lng = location.getLongitude(); ``` **C# (.NET):** ```csharp var locator = new Geolocator(); var position = await locator.GetGeopositionAsync(); var lat = position.Coordinate.Point.Position.Latitude; var lng = position.Coordinate.Point.Position.Longitude; ``` **Ruby:** ```ruby require 'geocoder' result = Geocoder.search("your-ip-address").first puts "Latitude: #{result.latitude}, Longitude: #{result.longitude}" ``` **Node.js:** ```javascript const geoip = require('geoip-lite'); const ip = "your-ip-address"; const geo = geoip.lookup(ip); console.log(`Latitude: ${geo.ll[0]}, Longitude: ${geo.ll[1]}`); ``` ## How to mock GPS location details with Selenium and Appium? **TestingBot** allows you to mock GPS location on both simulators/emulators and physical mobile devices during automated testing. This is essential for testing location-based features in your mobile applications. **Mocking GPS with Selenium (Chrome DevTools Protocol):** ```python # Python example with Selenium from selenium import webdriver driver = webdriver.Chrome() driver.execute_cdp_cmd("Emulation.setGeolocationOverride", { "latitude": 51.175570, "longitude": 3.945113, "accuracy": 100 }) ``` **Mocking GPS with Appium (iOS/Android):** ```python # Python example with Appium driver.set_location(51.175570, 3.945113, 10) # lat, lng, altitude ``` With TestingBot's real device cloud, you can: - Test location-based app features with custom coordinates - Simulate movement between locations for navigation testing - Test geofencing functionality - Verify location permissions handling [Learn more about TestingBot's real device testing](https://testingbot.com/real-device-testing) ## Popular GPS coordinates of famous places Here are GPS coordinates for some famous landmarks around the world: | Location | Latitude | Longitude | | --- | --- | --- | | Eiffel Tower, Paris | 48.8584 | 2.2945 | | Statue of Liberty, New York | 40.6892 | -74.0445 | | Big Ben, London | 51.5007 | -0.1246 | | Sydney Opera House | -33.8568 | 151.2153 | | Christ the Redeemer, Rio | -22.9519 | -43.2105 | | Machu Picchu, Peru | -13.1631 | -72.5450 | | Great Wall of China | 40.4319 | 116.5704 | | Taj Mahal, India | 27.1751 | 78.0421 | | Colosseum, Rome | 41.8902 | 12.4922 | | Burj Khalifa, Dubai | 25.1972 | 55.2744 | ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/px-to-rem-converter [Tools](https://testingbot.com/free-online-tools)Converters # PX to REM Converter Make your CSS responsive When designing modern web pages, switching from fixed pixel values (PX) to relative units like REM ensures your content adapts seamlessly across various devices and screen sizes. With just a few clicks, you can convert pixel values to REM units, making your design more flexible and accessible. ## PX to REM Converter Base size in px EM em Pixel px Rem rem Percentage % Point pt Values update automatically as you type ## What is the PX to REM Converter? The PX to REM Converter is a free online tool that helps web developers and designers convert pixel (PX) values to REM units. REM units are relative to the root element's font size, making them ideal for creating responsive and accessible web designs that scale properly across different devices and user preferences. This converter supports bidirectional conversion, allowing you to convert from pixels to REM or from REM to pixels, along with other related CSS units like EM, percentages, and points. ## Use Cases of the PX to REM Converter Tool - **Font Size Conversion:** Convert fixed pixel font sizes to scalable REM units for better accessibility - **Spacing and Margins:** Transform pixel-based spacing into responsive REM values for consistent layouts - **Component Design:** Ensure UI components scale properly across different screen sizes and user settings - **Legacy Code Migration:** Modernize existing CSS by converting pixel values to more flexible REM units - **Design System Implementation:** Establish consistent spacing and typography scales using REM values - **Accessibility Compliance:** Support users who adjust their browser's default font size for better readability ## What is the Difference Between PX and REM? **Pixels (PX):** Absolute units that represent a fixed size on the screen. 1px is always 1px regardless of the user's settings or device characteristics. While precise, pixels don't adapt to user preferences or different viewing contexts. **REM:** Relative units based on the root element's font size (typically 16px in most browsers). 1rem equals the root font size, so if a user increases their browser's font size, REM values scale proportionally, making content more accessible and responsive. The key advantage of REM is that it respects user preferences and creates more flexible, accessible designs that work well across different devices and accessibility needs. ## How to Use TestingBot's PX to REM Converter ### 1. Set the Root Font Size Enter your website's base font size in the "Base size in px" field. The default is 16px, which is the standard browser default. ### 2. Identify the PX Values You Want to Convert Look through your CSS for pixel values that you want to make more responsive, such as font sizes, margins, or padding. ### 3. Input Values into the Converter Enter your pixel values in the "Pixel" field, or REM values in the "Rem" field. The converter automatically calculates the equivalent values in real-time. ### 4. Replace PX with REM in Your CSS Copy the converted REM values and update your CSS files, replacing the fixed pixel values with the responsive REM equivalents. ### 5. Test for Browser Compatibility and Scaling Test your website by changing the browser's default font size to ensure your design scales properly and remains visually appealing. ## Is converting to REM necessary for responsive design? While not strictly necessary, converting to REM units significantly improves responsive design and accessibility. REM units automatically adapt to user preferences and different viewing contexts, making your website more inclusive and user-friendly. Modern web development best practices recommend using relative units like REM for better scalability and accessibility compliance. ## What's the default root font size in browsers? Most browsers use 16px as the default root font size. This means 1rem equals 16px by default. However, users can change this setting in their browser preferences, which is why using REM units is beneficial - your design will automatically adapt to these user preferences. ## When should I use REM instead of PX? Use REM for: - Font sizes and typography - Spacing (margins, padding) - Component dimensions that should scale with text - Any measurement that benefits from user accessibility settings Continue using PX for: - Border widths (1px borders) - Small decorative elements - Elements that must remain fixed regardless of font size ## Can I use both PX and REM in the same stylesheet? Yes, absolutely! It's common and recommended to use both PX and REM units in the same stylesheet. Use each unit where it makes the most sense - REM for scalable elements and PX for fixed elements. This hybrid approach gives you the flexibility of responsive design while maintaining precise control where needed. ## What does the PX to REM Converter do? The PX to REM Converter calculates the equivalent REM value for any given pixel measurement based on your specified root font size. It performs the simple calculation: REM = Pixels ÷ Root Font Size. The tool also provides conversions to related units like EM, percentages, and points, making it a comprehensive solution for CSS unit conversion. ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/og-meta-generator [Tools](https://testingbot.com/free-online-tools)Data Generators # Open Graph Generator Create social media meta tags Generate open-graph and social media HTML meta tags for your website. Optimize how your content appears when shared on Facebook, Twitter, LinkedIn and other social platforms. Page Type Website Article Product Profile Book Music Video Page URL Title Description Image URL Image Alt Text Image Width (px) Image Height (px) ## Generated Meta Tags Copy to Clipboard ## What are Open Graph meta tags? Open Graph meta tags are HTML meta elements that control how URLs are displayed when shared on social media platforms like Facebook, Twitter, LinkedIn, and others. These tags provide structured data about your webpage, including the title, description, image, and type of content. When you share a link on social media, the platform uses these Open Graph tags to create an attractive preview card with an image, title, and description. This greatly improves engagement and click-through rates compared to plain text links. ## What are the essential Open Graph meta tags? The four essential Open Graph meta tags that every webpage should have are: - **og:title** - The title of your page as it should appear in social media - **og:description** - A brief description of the page content - **og:image** - The URL to an image that represents your page - **og:url** - The canonical URL of your page Additionally, `og:type` helps specify the type of content (website, article, product, etc.) and `og:site_name` identifies your website or brand name. ## What image dimensions work best for Open Graph images? The recommended dimensions for Open Graph images are **1200x630 pixels** (1.91:1 aspect ratio). This size works optimally across all major social media platforms including Facebook, Twitter, LinkedIn, and others. - Minimum recommended size: 600x314 pixels - Maximum file size: Generally under 5MB for best performance - Supported formats: JPG, PNG, WebP, and GIF - Avoid images with too much text, as they may be penalized by some platforms ## How do I test my Open Graph meta tags? You can test your Open Graph meta tags using several free debugging tools: - **Facebook Sharing Debugger** - Official tool from Facebook to preview and debug Open Graph tags - **Twitter Card Validator** - Twitter's tool for validating Twitter Card meta tags - **LinkedIn Post Inspector** - LinkedIn's tool for previewing how posts will appear - **Open Graph Check** - Third-party tools that validate multiple platforms at once These tools will show you exactly how your page will appear when shared and highlight any missing or problematic tags. ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/qr-code-generator [Tools](https://testingbot.com/free-online-tools)Data Generators # QR Code Generator Create scannable QR codes Generate and download QR codes for URLs, text, or any content. Customize colors and error correction levels for optimal scanning performance. Text or URL Foreground Color Background Color Error Correction Level Low (~7%) Medium (~15%) Quartile (~25%) High (~30%) Higher levels allow the QR code to be read even if partially damaged or obscured. ## Generated QR Code Download QR Code ## What is a QR Code? A QR Code (Quick Response Code) is a two-dimensional barcode that can store various types of information such as text, URLs, phone numbers, or other data. QR codes can be quickly scanned by smartphones, tablets, and dedicated QR code readers to instantly access the encoded information. Originally developed in 1994 for tracking automotive parts, QR codes have become widely used for marketing, payments, authentication, and information sharing due to their fast readability and large storage capacity compared to traditional barcodes. ## What can I encode in a QR Code? QR codes can encode various types of data including: - **URLs and Websites** - Direct users to web pages, social media profiles, or online content - **Plain Text** - Any text information, messages, or instructions - **Contact Information** - Phone numbers, email addresses, or vCard data - **WiFi Credentials** - Network name and password for easy connection - **SMS Messages** - Pre-filled text messages with recipient numbers - **Email Addresses** - mailto links with optional subject and body text - **Geographic Locations** - GPS coordinates or addresses for mapping applications ## What do the error correction levels mean? Error correction levels determine how much of the QR code can be damaged or obscured while still remaining readable: - **Low (L)** - Can recover from about 7% damage. Best for clean, controlled environments. - **Medium (M)** - Can recover from about 15% damage. Good balance for most applications. - **Quartile (Q)** - Can recover from about 25% damage. Better for outdoor or rough handling. - **High (H)** - Can recover from about 30% damage. Best for harsh environments or when partial obscuring is likely. Higher error correction levels create larger QR codes with more data redundancy, so choose the level that best matches your intended use case. ## How do I scan a QR Code? Most modern smartphones can scan QR codes without additional apps: - **iPhone (iOS 11+)** - Open the Camera app and point it at the QR code. A notification will appear with the content. - **Android** - Open the Camera app or Google Assistant, point at the QR code. Many Android phones have built-in QR scanning in their camera apps. - **Dedicated Apps** - Download QR code scanner apps from app stores for additional features like scan history and batch processing. - **Web Browsers** - Some browsers like Chrome and Edge have built-in QR code scanning capabilities. For best scanning results, ensure good lighting, hold the device steady, and position the QR code within the camera's viewfinder. ## What are the best practices for QR Code design? Follow these best practices to ensure your QR codes scan reliably: - **Contrast** - Use high contrast between foreground and background colors (dark on light works best) - **Size** - Minimum size of 2x2 cm (0.8x0.8 inches) for print, larger for distance scanning - **Quiet Zone** - Leave white space around the QR code (at least 4 modules wide) - **Testing** - Always test your QR codes on multiple devices before deploying - **Content Length** - Shorter content creates simpler, more scannable QR codes - **Print Quality** - Use high-resolution images for printing to avoid pixelation ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/device-information [Tools](https://testingbot.com/free-online-tools)Data Generators # Device Information Detect browser and system details Get comprehensive information about your device, browser, screen resolution, operating system, and capabilities. Perfect for testing and development. ## Browser Information Browser Name Detecting... Browser Version Detecting... Browser Engine Detecting... JavaScript Enabled Yes Cookies Enabled Detecting... Local Storage Detecting... User Agent Detecting... ## Operating System Operating System Detecting... Platform Detecting... Architecture Detecting... CPU Cores Detecting... Memory (Approx) Detecting... Device Type Detecting... ## Screen & Display Screen Resolution Detecting... Available Resolution Detecting... Browser Window Size Detecting... Viewport Size Detecting... Color Depth Detecting... Pixel Ratio Detecting... Orientation Detecting... Touch Support Detecting... ## Network & Connection Connection Type Detecting... Effective Type Detecting... Downlink Speed Detecting... RTT (Ping) Detecting... Online Status Detecting... Do Not Track Detecting... ## Browser Capabilities WebGL Support Detecting... WebRTC Support Detecting... Geolocation API Detecting... Service Worker Detecting... WebSocket Support Detecting... IndexedDB Support Detecting... Canvas Support Detecting... WebAssembly Detecting... ## Languages & Time Languages Detecting... Timezone Detecting... Current Time Detecting... UTC Offset Detecting... ## What information can this tool detect about my device? This device information tool can detect a wide range of details about your system including: - **Browser Information** - Name, version, engine, user agent, and capabilities - **Operating System** - OS name, platform, architecture, CPU cores, and memory - **Screen Details** - Resolution, color depth, pixel ratio, and orientation - **Network Information** - Connection type, speed, and online status - **Supported Technologies** - WebGL, WebRTC, Canvas, WebAssembly support - **Language & Time** - Browser languages, timezone, and current time ## Why would I need to check my device information? Checking device information is useful for several purposes: - **Web Development** - Testing how your website appears on different devices and browsers - **Troubleshooting** - Identifying browser compatibility issues or missing features - **System Diagnostics** - Checking if your browser supports modern web technologies - **Privacy Awareness** - Understanding what information websites can detect about you - **Technical Support** - Providing detailed system information when reporting issues - **Performance Testing** - Understanding device capabilities for optimization ## How accurate is browser and device detection? Device detection accuracy varies depending on the type of information being detected: - **Highly Accurate** - Screen resolution, browser capabilities, JavaScript/cookie support - **Generally Accurate** - Browser name/version, basic OS detection, language preferences - **Approximate** - Device memory, CPU cores (limited by browser security policies) - **May Be Blocked** - Geolocation, device orientation (requires user permission) Some information may be limited or blocked by privacy settings, browser extensions, or security policies. Modern browsers intentionally limit certain device fingerprinting capabilities to protect user privacy. ## Can websites track me using this device information? While device information can be used for tracking, there are important privacy considerations: - **Browser Fingerprinting** - Combinations of device characteristics can create unique identifiers - **Privacy Protection** - Modern browsers limit access to sensitive device information - **User Control** - You can disable JavaScript, use privacy extensions, or browse in incognito mode - **Legitimate Uses** - Most websites use device info for optimization and compatibility, not tracking To enhance privacy, consider using browser privacy settings, VPNs, or privacy-focused browsers that limit device fingerprinting capabilities. ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/user-agent-parser [Tools](https://testingbot.com/free-online-tools)Data Generators # User Agent Parser Analyze browser user agents Analyze and parse user agent strings to extract detailed information about browsers, engines, operating systems, CPU architecture, and devices. ## User Agent String Enter or paste a user agent string to analyze: Your current user agent is automatically loaded Parse User Agent ## Browser Name - Version - Major Version - ## Engine Name - Version - ## Operating System Name - Version - ## Device Type - Model - Vendor - ## CPU Architecture - ## Top 10 Most Popular User Agents (2026) | Browser | OS | Device | Action | | --- | --- | --- | --- | ## What is a User Agent? A User Agent is a string of text that web browsers and applications send to web servers to identify themselves. It contains crucial information about the browser name, version, operating system, device type, and rendering engine being used. This identification helps web servers deliver optimized content for specific browsers and devices. The User Agent string follows a semi-standardized format but can vary significantly between browsers. It typically includes: - Browser name and version (e.g., Chrome/120.0.0.0) - Operating system information (e.g., Windows NT 10.0) - Rendering engine details (e.g., Gecko, WebKit, Blink) - Device information for mobile devices - Additional compatibility tokens Understanding User Agent strings is essential for web development, browser compatibility testing, analytics, and providing optimal user experiences across different platforms. ## What are the Most Popular User Agents? The most popular User Agents in 2024 are primarily from Chrome, Safari, Firefox, and Edge browsers across desktop and mobile platforms. Here are the key trends: - **Chrome on Windows** - The most common desktop combination, typically Windows 10/11 with Chrome 120+ - **Safari on iPhone** - Dominant mobile browser on iOS devices, usually the latest iOS versions - **Chrome on Android** - Leading Android browser across various device manufacturers - **Safari on macOS** - Popular among Mac users, especially developers and designers - **Edge on Windows** - Growing market share as Microsoft's default browser - **Firefox** - Maintains steady usage among privacy-conscious users Mobile User Agents now account for over 60% of web traffic globally, with Chrome and Safari dominating. Desktop usage remains strong for productivity and professional applications. The diversity of User Agents continues to grow with new devices like smart TVs, gaming consoles, and IoT devices accessing web content. ## How Can I Change My User Agent? Changing your User Agent can be useful for testing, development, or accessing content optimized for different browsers. Here are several methods: ### Browser Developer Tools Most modern browsers allow temporary User Agent changes: - **Chrome/Edge** : Open DevTools (F12) → Network Conditions → User Agent → Uncheck "Use browser default" - **Firefox** : Open DevTools (F12) → Responsive Design Mode → Settings → Custom User Agent - **Safari** : Enable Develop menu → User Agent → Select or enter custom ### Browser Extensions Popular extensions for permanent User Agent switching: - User-Agent Switcher and Manager - User-Agent Switcher for Chrome - Random User-Agent ### Command Line Browsers ``` # Chrome with custom User Agent chrome --user-agent="Custom User Agent String" # cURL with custom User Agent curl -H "User-Agent: Custom User Agent" https://testingbot.com ``` **Note:** Changing your User Agent may affect website functionality as some sites deliver different content based on the detected browser. Always test thoroughly when using custom User Agents in production environments. ## How to Get User Agent in Different Programming Languages _JavaScript Browser & Node.js_ ```javascript // Browser JavaScript const userAgent = navigator.userAgent; console.log('User Agent:', userAgent); // Node.js - Getting client User Agent in Express const express = require('express'); const app = express(); app.get('/', (req, res) => { const userAgent = req.headers['user-agent']; res.json({ userAgent }); }); // Setting custom User Agent in Node.js fetch fetch('https://api.testingbot.com', { headers: { 'User-Agent': 'MyApp/1.0.0' } }); ``` _Python Flask & Requests_ ```python # Flask - Getting client User Agent from flask import Flask, request app = Flask( __name__ ) @app.route('/') def index(): user_agent = request.headers.get('User-Agent') return {'user_agent': user_agent} # Using requests library with custom User Agent import requests headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/120.0.0.0' } response = requests.get('https://testingbot.com', headers=headers) # Getting User Agent from HTTP request from http.server import BaseHTTPRequestHandler class Handler(BaseHTTPRequestHandler): def do_GET(self): user_agent = self.headers.get('User-Agent') print(f"User Agent: {user_agent}") ``` _PHP Server & cURL_ ```php [ 'header' => "User-Agent: MyApp/1.0\r\n" ] ]; $context = stream_context_create($options); $content = file_get_contents('https://testingbot.com', false, $context); ?> ``` _Ruby Rails & Net::HTTP_ ```ruby # Rails - Getting client User Agent class ApplicationController < ActionController::Base def index user_agent = request.user_agent render json: { user_agent: user_agent } end end # Using Net::HTTP with custom User Agent require 'net/http' require 'uri' uri = URI('https://testingbot.com') http = Net::HTTP.new(uri.host, uri.port) http.use_ssl = true request = Net::HTTP::Get.new(uri) request['User-Agent'] = 'MyRubyApp/1.0' response = http.request(request) # Using HTTParty gem require 'httparty' response = HTTParty.get('https://testingbot.com', headers: { 'User-Agent' => 'MyApp/1.0' } ) ``` _Java Servlet & HttpClient_ ```java // Servlet - Getting client User Agent import javax.servlet.http.HttpServletRequest; public class UserAgentServlet extends HttpServlet { protected void doGet(HttpServletRequest request, HttpServletResponse response) { String userAgent = request.getHeader("User-Agent"); System.out.println("User Agent: " + userAgent); } } // Using HttpURLConnection URL url = new URL("https://testingbot.com"); HttpURLConnection conn = (HttpURLConnection) url.openConnection(); conn.setRequestProperty("User-Agent", "MyJavaApp/1.0"); // Using Apache HttpClient CloseableHttpClient client = HttpClients.custom() .setUserAgent("MyApp/1.0") .build(); HttpGet request = new HttpGet("https://testingbot.com"); CloseableHttpResponse response = client.execute(request); // Spring Boot @RestController public class UserAgentController { @GetMapping("/") public String getUserAgent(@RequestHeader("User-Agent") String userAgent) { return userAgent; } } ``` _C# ASP.NET & HttpClient_ ```csharp // ASP.NET Core - Getting client User Agent public class HomeController : Controller { public IActionResult Index() { string userAgent = Request.Headers["User-Agent"].ToString(); return Json(new { userAgent }); } } // Using HttpClient with custom User Agent using var client = new HttpClient(); client.DefaultRequestHeaders.UserAgent.ParseAdd("MyApp/1.0"); var response = await client.GetAsync("https://testingbot.com"); // WebClient (legacy) using var webClient = new WebClient(); webClient.Headers.Add("User-Agent", "MyApp/1.0"); string result = webClient.DownloadString("https://testingbot.com"); // ASP.NET Core Middleware public class UserAgentMiddleware { public async Task InvokeAsync(HttpContext context) { var userAgent = context.Request.Headers["User-Agent"].ToString(); Console.WriteLine($"User Agent: {userAgent}"); await _next(context); } } ``` _Go net/http_ ```go package main import ( "fmt" "net/http" ) // HTTP Server - Getting client User Agent func handler(w http.ResponseWriter, r *http.Request) { userAgent := r.Header.Get("User-Agent") fmt.Fprintf(w, "User Agent: %s", userAgent) } func main() { http.HandleFunc("/", handler) http.ListenAndServe(":8080", nil) } // HTTP Client - Setting custom User Agent client := &http.Client{} req, _ := http.NewRequest("GET", "https://testingbot.com", nil) req.Header.Set("User-Agent", "MyGoApp/1.0") resp, err := client.Do(req) if err != nil { log.Fatal(err) } defer resp.Body.Close() ``` _Perl CGI & LWP_ ```perl #!/usr/bin/perl use strict; use warnings; # CGI - Getting client User Agent use CGI; my $q = CGI->new; my $user_agent = $q->user_agent(); print "User Agent: $user_agent\n"; # Using LWP::UserAgent use LWP::UserAgent; my $ua = LWP::UserAgent->new; $ua->agent("MyPerlBot/1.0"); my $response = $ua->get('https://testingbot.com'); if ($response->is_success) { print $response->decoded_content; } # Using HTTP::Tiny use HTTP::Tiny; my $http = HTTP::Tiny->new( agent => 'MyPerlApp/1.0' ); my $response = $http->get('https://testingbot.com'); ``` _Objective-C iOS & macOS_ ```objectivec // Getting system User Agent in iOS/macOS #import // Method 1: Using UIWebView (deprecated) UIWebView *webView = [[UIWebView alloc] init]; NSString *userAgent = [webView stringByEvaluatingJavaScriptFromString:@"navigator.userAgent"]; NSLog(@"User Agent: %@", userAgent); // Method 2: Using WKWebView #import WKWebView *webView = [[WKWebView alloc] init]; [webView evaluateJavaScript:@"navigator.userAgent" completionHandler:^(NSString *userAgent, NSError *error) { NSLog(@"User Agent: %@", userAgent); }]; // Setting custom User Agent with NSURLRequest NSURL *url = [NSURL URLWithString:@"https://testingbot.com"]; NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:url]; [request setValue:@"MyIOSApp/1.0" forHTTPHeaderField:@"User-Agent"]; // Using NSURLSession NSURLSession *session = [NSURLSession sharedSession]; NSURLSessionDataTask *task = [session dataTaskWithRequest:request completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) { // Handle response }]; [task resume]; ``` _Swift iOS & macOS_ ```swift import UIKit import WebKit // Getting User Agent with WKWebView let webView = WKWebView() webView.evaluateJavaScript("navigator.userAgent") { (result, error) in if let userAgent = result as? String { print("User Agent: \(userAgent)") } } // Setting custom User Agent with URLRequest var request = URLRequest(url: URL(string: "https://testingbot.com")!) request.setValue("MySwiftApp/1.0", forHTTPHeaderField: "User-Agent") // Using URLSession let task = URLSession.shared.dataTask(with: request) { data, response, error in // Handle response } task.resume() // SwiftUI with custom User Agent struct WebView: UIViewRepresentable { func makeUIView(context: Context) -> WKWebView { let webView = WKWebView() webView.customUserAgent = "MyCustomApp/1.0" return webView } } ``` _Rust Actix-web & Reqwest_ ```rust // Actix-web - Getting client User Agent use actix_web::{web, App, HttpRequest, HttpServer, Result}; async fn index(req: HttpRequest) -> Result { let user_agent = req.headers() .get("user-agent") .and_then(|h| h.to_str().ok()) .unwrap_or("Unknown"); Ok(format!("User Agent: {}", user_agent)) } #[actix_web::main] async fn main() -> std::io::Result<()> { HttpServer::new(|| App::new().route("/", web::get().to(index))) .bind("127.0.0.1:8080")? .run() .await } // Using Reqwest with custom User Agent use reqwest; #[tokio::main] async fn main() -> Result<(), Box> { let client = reqwest::Client::builder() .user_agent("MyRustApp/1.0") .build()?; let response = client .get("https://testingbot.com") .send() .await?; Ok(()) } ``` ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/image-extractor [Tools](https://testingbot.com/free-online-tools)Data Generators # Image Extractor Extract images from any website Extract all images from any public website including background images, dynamically loaded images and SVG elements. Find and download images in various formats including JPEG, PNG, WebP, SVG and more from any webpage. Enter URL: ![Extracting Images](https://testingbot.com/assets/loading-0406eae1b17f4903931d9817122fdfbe585651d8fb63e564cf9def4c609fb9f7.gif) ## Error ## Extracted Images: This image extraction tool finds all images on a webpage, including background images, dynamically loaded images and embedded SVG elements. It provides detailed information about each image including dimensions, file type and download options. The tool is perfect for web designers, developers and content creators who need to analyze or collect images from websites for various purposes. ## What Is the Image Extractor Tool? The Image Extractor Tool is a powerful web-based utility that automatically discovers and extracts all images from any public website. Unlike simple image scrapers, this tool uses advanced browser automation to find images that are dynamically loaded, set as CSS background images, or embedded as SVG elements. The tool provides comprehensive image analysis including file format detection, dimension measurements and direct download capabilities for each discovered image. ## What Are the Use Cases for This Tool? This image extraction tool has numerous practical applications across different fields: - **Web Design & Development:** Analyze competitor websites, extract design assets or audit image usage across web properties - **Content Creation:** Gather reference images, mood boards or inspiration from various websites - **Digital Marketing:** Research visual trends, analyze competitor imagery or collect assets for campaigns - **Research & Analysis:** Academic research, market analysis or visual content studies - **Quality Assurance:** Test image loading, validate image formats or audit website assets This free tool supports all major image formats including `JPEG`, `PNG`, `WebP`, `SVG`, `GIF` and more, making it versatile for any image extraction needs. ## How to Use the Tool? Using the Image Extractor Tool is straightforward: 1. Enter the URL of the website you want to extract images from in the input field 2. Click the "Extract Images" button to start the extraction process 3. Wait for the tool to analyze the webpage and discover all images 4. Browse the results in the image grid, showing thumbnails with image details 5. Click the download button on any image to save it to your device The tool will display each image with its dimensions, file type and file size, making it easy to identify the images you need. ## What Are the Advantages of This Tool? Our Image Extractor Tool offers several key advantages over manual image collection or basic scrapers: - **Comprehensive Detection:** Finds all images including background images, lazy-loaded content and dynamically generated images - **Multiple Format Support:** Handles JPEG, PNG, WebP, SVG, GIF and other modern image formats - **Detailed Information:** Provides image dimensions, file size and format details for each discovered image - **Browser Automation:** Uses real browser rendering to ensure all images are discovered, even those loaded by JavaScript - **Direct Downloads:** One-click download functionality for any extracted image - **No Installation Required:** Works entirely in your web browser without any software installation - **Privacy Focused:** Images are processed securely without storing or caching your data The tool is perfect for professionals who need reliable, comprehensive image extraction with detailed metadata. ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/unix-timestamp-converter [Tools](https://testingbot.com/free-online-tools)Time & Date # Unix Timestamp Converter Convert epoch time instantly Convert Unix timestamps to human-readable dates and vice versa. Get the current Unix timestamp with real-time updates. ## Current Unix Timestamp 0 Seconds since Jan 01 1970 (UTC) Current time Copy Timestamp ## Convert Timestamp to Date Enter Unix timestamp Convert **GMT:** **Your Time Zone:** **Relative:** ## Convert Date to Timestamp Year Month January February March April May June July August September October November December Day Hour (24hr) Minutes Seconds Convert to Unix Timestamp **Unix Timestamp:** **GMT:** **Your Time Zone:** **Relative:** ## Current Epoch Information **Date:** **UTC:** **ISO 8601:** **RFC 822:** **RFC 2822:** **RFC 3339:** RFC Standards: [RFC 822](https://tools.ietf.org/html/rfc822), [RFC 1036](https://tools.ietf.org/html/rfc1036), [RFC 1123](https://tools.ietf.org/html/rfc1123), [RFC 2822](https://tools.ietf.org/html/rfc2822), [RFC 3339](https://tools.ietf.org/html/rfc3339) ## What is the Unix timestamp? The Unix timestamp (also known as Unix time, POSIX time, or epoch time) is a system for describing points in time: the number of seconds that have elapsed since January 1, 1970, at 00:00:00 UTC (Coordinated Universal Time). This date is known as the Unix epoch. Unix timestamps are widely used in programming and computing because they provide a simple, universal way to represent time that is independent of time zones and calendar systems. They are stored as integers, making them easy to work with in databases and programming languages. ## Why should I use a Unix timestamp? Unix timestamps offer several advantages for developers and system administrators: - **Universal Standard:** Works across all time zones and doesn't require time zone conversion - **Simplicity:** Stored as a single integer value, making calculations and comparisons easy - **Efficiency:** Takes up minimal storage space compared to formatted date strings - **Sorting:** Natural chronological ordering when sorted numerically - **Database-friendly:** Most databases have built-in functions to work with Unix timestamps - **API Integration:** Many APIs use Unix timestamps for date/time data exchange ## What happens on January 19, 2038? January 19, 2038, at 03:14:07 UTC marks the "Year 2038 problem" or "Y2038 bug." This is when 32-bit signed integer Unix timestamps will overflow, rolling over from 2,147,483,647 to -2,147,483,648, which would be interpreted as December 13, 1901. This issue affects systems that use 32-bit integers to store Unix timestamps. However, most modern systems have already migrated to 64-bit timestamps, which won't have this problem for approximately 292 billion years. The transition to 64-bit systems and updated software has largely mitigated this concern for contemporary applications. ## How to get Unix timestamp in programming languages ### Get current Unix timestamp: **JavaScript:** `Math.floor(Date.now() / 1000)` **PHP:** `time()` **Python:** `int(time.time())` **Ruby:** `Time.now.to_i` **Java:** `System.currentTimeMillis() / 1000` **C#:** `DateTimeOffset.UtcNow.ToUnixTimeSeconds()` **Go:** `time.Now().Unix()` **Bash:** `date +%s` **MySQL:** `SELECT UNIX_TIMESTAMP()` **PostgreSQL:** `SELECT EXTRACT(EPOCH FROM NOW())` ### Convert Unix timestamp to human-readable date: **JavaScript:** `new Date(timestamp * 1000).toString()` **PHP:** `date('Y-m-d H:i:s', $timestamp)` **Python:** `datetime.fromtimestamp(timestamp)` **Ruby:** `Time.at(timestamp)` **Java:** `new Date(timestamp * 1000)` **C#:** `DateTimeOffset.FromUnixTimeSeconds(timestamp)` **Go:** `time.Unix(timestamp, 0)` **Bash:** `date -d @timestamp` **MySQL:** `FROM_UNIXTIME(timestamp)` **PostgreSQL:** `TO_TIMESTAMP(timestamp)` ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/cron-expression-generator [Tools](https://testingbot.com/free-online-tools)Time & Date # Cron Expression Generator Build and test cron schedules This free tool allows you to enter a cron expression and get back a human-readable description along with the next 10 run times. You can also build a cron expression using the builder interface. ## Cron Expression Tester Enter Cron Expression Parse Format: minute hour day month weekday Quick Examples: Every minute Every hour Daily at midnight Weekly on Monday Monthly on 1st Every 5 minutes Every hour 9am-5pm weekdays Yearly on Jan 1st ## Expression Details Human Readable Daily at midnight Next 10 Run Times Field Breakdown 0 Minute At minute 0 0 Hour At hour 0 \* Day Every day \* Month Every month \* Weekday Every weekday ## Cron Expression Builder Minute (0-59) Every minute (\*) At minute 0 Every 5 minutes (\*/5) Every 10 minutes (\*/10) Every 15 minutes (\*/15) Every 30 minutes (\*/30) At minute 0 and 30 Custom... Hour (0-23) Every hour (\*) At midnight (0) Every 2 hours (\*/2) Every 4 hours (\*/4) Business hours (9-17) At midnight and noon Custom... Day of Month (1-31) Every day (\*) First day (1) Middle of month (15) Last day (L) Every 2 days (\*/2) Custom... Month (1-12) Every month (\*) January (1) Every quarter (\*/3) Every 6 months (\*/6) Custom... Day of Week (0-6, 0=Sunday) Every day (\*) Weekdays (1-5) Weekend (0,6) Monday (1) Friday (5) Custom... Generate Expression Generated Expression: Copy ## What is a Cron? Cron is a time-based job scheduler in Unix-like operating systems. It enables users to schedule jobs (commands or scripts) to run automatically at specified times, dates, or intervals. Cron expressions are strings that define when and how often a scheduled task should run. A cron expression consists of five fields separated by spaces: minute (0-59), hour (0-23), day of month (1-31), month (1-12), and day of week (0-6, where 0 is Sunday). Special characters like asterisk (\*) for "any value", comma (,) for multiple values, dash (-) for ranges and slash (/) for step values provide flexibility in scheduling. ## How can I use crons for scheduling? Cron expressions are widely used for scheduling automated tasks across various platforms and applications: - **System Administration:** Schedule backups, log rotation, system updates and maintenance tasks - **Web Applications:** Trigger periodic data processing, send scheduled emails or generate reports - **CI/CD Pipelines:** Run automated builds, tests, or deployments on a schedule - **Monitoring:** Execute health checks, collect metrics, or run diagnostic scripts at regular intervals - **Data Processing:** Schedule ETL jobs, data synchronization or batch processing tasks Common cron patterns include: "0 2 \* \* \*" for daily at 2 AM, "0 \*/4 \* \* \*" for every 4 hours, "0 9 \* \* 1-5" for weekdays at 9 AM, and "\*/15 \* \* \* \*" for every 15 minutes. ## TestingBot allows adding codeless tests with cron format to do scheduled testing TestingBot's [codeless testing platform](https://testingbot.com/features/ai-testing) integrates cron expressions for powerful scheduled test automation. You can set up your tests to run automatically at specified intervals without writing any code, ensuring continuous validation of your web applications and APIs. With TestingBot's cron-based scheduling, you can: - **Run regression tests nightly:** Use "0 2 \* \* \*" to execute your test suite at 2 AM daily when traffic is low - **Monitor critical paths hourly:** Set "0 \* \* \* \*" to verify essential user journeys every hour - **Weekend maintenance checks:** Schedule "0 10 \* \* 6,0" for Saturday and Sunday morning validations - **Get instant alerts:** Receive notifications via email, Slack, or webhooks when scheduled tests fail - **Track test history:** Review detailed reports and trends from all scheduled test runs This automated approach helps detect issues early, reduces manual testing effort, and ensures your application remains functional 24/7. Simply create your test scenario using TestingBot's codeless recorder, add a cron schedule, and let the platform handle the rest. ## Cron Expression Reference ### Field Values | Field | Allowed Values | Special Characters | | --- | --- | --- | | Minute | 0-59 | , - \* / | | Hour | 0-23 | , - \* / | | Day of Month | 1-31 | , - \* / L W | | Month | 1-12 or JAN-DEC | , - \* / | | Day of Week | 0-6 or SUN-SAT | , - \* / L # | ### Special Characters **\* (asterisk):** Matches any value. "\* \* \* \* \*" runs every minute **, (comma):** Separates multiple values. "0,30 \* \* \* \*" runs at minute 0 and 30 **- (dash):** Defines a range. "9-17 \* \* \* \*" runs from 9 AM to 5 PM **/ (slash):** Defines step values. "\*/15 \* \* \* \*" runs every 15 minutes **L:** Last day/weekday of month. "0 0 L \* \*" runs on the last day of the month **W:** Nearest weekday. "0 0 15W \* \*" runs on the weekday nearest to the 15th **#:** Nth occurrence. "0 0 \* \* 5#2" runs on the second Friday ### Common Examples `0 0 * * *` - Daily at midnight `0 */6 * * *` - Every 6 hours `0 0 * * 1` - Weekly on Monday `0 0 1 * *` - Monthly on the 1st `0 0 1 1 *` - Yearly on January 1st `*/30 9-17 * * 1-5` - Every 30 minutes during business hours on weekdays ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/whats-my-ip-address [Tools](https://testingbot.com/free-online-tools)Network Tools # What's My IP Address? Find your public IP instantly Find out your public IP address, location, ISP, and other network information instantly with our free IP lookup tool. ## Your IP Address Information Detecting your IP address... ### Your Public IP Address IPv4 IPv6 Copy IP Address ### Network Information Decimal: - Hostname: - ASN: - ISP: - Services: - ### Location Information Country: - State/Region: - City: - Postal Code: - Timezone: - Latitude: - Longitude: - ### Your Location on Map ### Browser Information User Agent: - Language: - Referrer: - ## What's my IP address? Your IP address is a unique identifier assigned to your device when it connects to the internet. This tool shows you both your IPv4 and IPv6 addresses, along with detailed information about your network connection and approximate location. The IP address displayed here is your **public IP address** - the one that websites and online services see when you connect to them. This is different from your private IP address used within your local network. ## What is a public IP address? A public IP address is a globally unique address that identifies your device on the internet. Unlike private IP addresses (used within your home or office network), public IP addresses are assigned by your Internet Service Provider (ISP) and are visible to websites and online services. Key characteristics of public IP addresses: - **Globally unique:** No two devices on the internet can have the same public IP address at the same time - **ISP assigned:** Your internet provider assigns and manages your public IP address - **Location identifiable:** Public IP addresses can be used to determine your approximate geographic location - **Dynamic or static:** Most home users have dynamic IPs that change periodically, while businesses often have static IPs ## How does my public IP address affect me? Your public IP address affects your online experience in several ways: - **Geographic restrictions:** Some websites and streaming services use your IP to determine your location and may restrict content based on your country - **Website analytics:** Websites use IP addresses to track visitor statistics, detect fraud, and provide location-specific content - **Online security:** Your IP can be used to identify suspicious activity or block malicious traffic - **Network troubleshooting:** IT administrators use IP addresses to diagnose and resolve network issues - **Privacy concerns:** Your IP address can potentially be used to track your online activities across different websites ## Should I change my IP address? Whether you should change your IP address depends on your specific needs and concerns: **Reasons you might want to change your IP:** - Access geo-restricted content from other regions - Enhance privacy and reduce online tracking - Bypass network restrictions or censorship - Resolve connectivity issues with certain websites - Protect against DDoS attacks or harassment **Methods to change your IP address:** - Restart your modem/router (may work for dynamic IPs) - Contact your ISP to request a new IP address - Use a VPN (Virtual Private Network) service - Use a proxy server - Use mobile data instead of your home internet ## How do I show my IP address? There are several ways to find and display your IP address: **Online tools (like this one):** - Visit websites that automatically detect and display your public IP - These tools often provide additional information like location and ISP details - Quick and easy method that works on any device with internet access **Command line methods:** - **Windows:** `ipconfig` (local IP) or `nslookup myip.opendns.com resolver1.opendns.com` (public IP) - **Mac/Linux:** `ifconfig` (local IP) or `curl ifconfig.me` (public IP) - **Universal:** `curl ipinfo.io` for detailed IP information ## What are proxies, how do they change my IP? A proxy server acts as an intermediary between your device and the internet. When you use a proxy, your internet traffic is routed through the proxy server, which then communicates with websites on your behalf. **How proxies change your IP address:** - Websites see the proxy server's IP address instead of your real IP - Your actual IP address is hidden from the websites you visit - The proxy server's location determines what websites think is your location **Types of proxies:** - **HTTP proxies:** Work with web browsers and HTTP traffic - **SOCKS proxies:** Handle various types of traffic, more versatile - **Transparent proxies:** Don't hide your IP, mainly used for caching - **Anonymous proxies:** Hide your IP but identify themselves as proxies - **Elite proxies:** Hide your IP and don't identify as proxies **Proxy vs VPN:** While both change your apparent IP address, VPNs encrypt your entire internet connection and typically offer better security and privacy than basic proxy servers. ## Technical References Internet Protocol Standards: [RFC 791 (IPv4)](https://tools.ietf.org/html/rfc791), [RFC 2460 (IPv6)](https://tools.ietf.org/html/rfc2460), [RFC 1918 (Private Networks)](https://tools.ietf.org/html/rfc1918), [RFC 6890 (Special-Use IP Addresses)](https://tools.ietf.org/html/rfc6890) ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/ip-subnet-calculator [Tools](https://testingbot.com/free-online-tools)Network Tools # IP Subnet Calculator Calculate subnets, CIDR, netmasks and host ranges Enter an IPv4 address with a subnet mask or CIDR prefix to instantly calculate the network address, broadcast address, usable host range, wildcard mask and more. Supports all subnet sizes from /0 to /32. ## IP Subnet Calculator IP Address Subnet Mask / CIDR Calculate Show all subnets in this network | Network Address | Usable Range | Broadcast | | --- | --- | --- | ### CIDR / Subnet Reference Table | CIDR | Subnet Mask | Wildcard Mask | Total Addresses | Usable Hosts | | --- | --- | --- | --- | --- | ## What is an IP Subnet Calculator? An IP subnet calculator is a tool that takes an IPv4 address and a subnet mask (or CIDR prefix length) and computes key network properties: the network address, broadcast address, usable host range, wildcard mask, and number of available hosts. It is essential for network engineers, system administrators, and developers who need to plan IP addressing schemes, configure firewalls, or set up routing tables. ## What is CIDR notation? CIDR (Classless Inter-Domain Routing) notation is a compact way to represent an IP address and its associated subnet mask. It is written as `IP/prefix`, for example `192.168.1.0/24`. The number after the slash indicates how many leading bits of the address are used for the network portion. A /24 means the first 24 bits are the network part, leaving 8 bits (256 addresses) for hosts. CIDR replaced the older classful addressing system (Class A, B, C) and allows for more flexible allocation of IP address space. ## What is the difference between a subnet mask and a wildcard mask? A subnet mask and a wildcard mask are inverses of each other. The subnet mask has 1-bits for the network portion and 0-bits for the host portion (e.g. `255.255.255.0` for a /24). The wildcard mask flips this: 0-bits for the network and 1-bits for the host portion (e.g. `0.0.0.255`). Wildcard masks are commonly used in Cisco ACLs (Access Control Lists) and OSPF routing configurations, while subnet masks are used in most other networking contexts. ## How do I calculate the number of usable hosts in a subnet? The formula is `2^(32 - prefix) - 2` for subnets with a prefix of /30 or less. The two addresses subtracted are the network address (first address) and the broadcast address (last address), which cannot be assigned to hosts. For a /31, RFC 3021 allows both addresses to be used as a point-to-point link (2 usable). A /32 is a single host address (1 usable). For example, a /24 subnet has 2^8 - 2 = 254 usable host addresses. ## What are the common subnet sizes? The most commonly used subnet sizes are: - **/8** (255.0.0.0) — 16,777,214 hosts — large enterprise or ISP networks - **/16** (255.255.0.0) — 65,534 hosts — medium to large networks - **/24** (255.255.255.0) — 254 hosts — the most common LAN subnet - **/25** (255.255.255.128) — 126 hosts — splitting a /24 in half - **/26** (255.255.255.192) — 62 hosts — smaller office networks - **/27** (255.255.255.224) — 30 hosts — small workgroups - **/28** (255.255.255.240) — 14 hosts — very small segments - **/30** (255.255.255.252) — 2 hosts — point-to-point links - **/32** (255.255.255.255) — 1 host — single host route ## How is subnet calculation used in testing? When testing network-dependent applications, you often need to configure test environments with specific subnet settings. A subnet calculator helps you quickly determine the correct network addresses, gateway IPs, and host ranges for test VMs, containers, or cloud instances. It is also useful when writing automated tests that validate firewall rules, ACLs, or IP whitelisting logic — you can verify which IPs fall within a given subnet range. ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/css-prettify [Tools](https://testingbot.com/free-online-tools)Code Formatter # CSS Prettifier Format, beautify or minify CSS instantly Paste your CSS code below to instantly format it with proper indentation and line breaks, or minify it to reduce file size. Supports custom indent sizes with spaces or tabs. ## CSS Prettifier & Minifier Indent 2 spaces 4 spaces Tab Prettify Minify Copy ### Enter CSS ### Output ## What is a CSS Prettifier? A CSS prettifier (also called CSS beautifier or CSS formatter) is a tool that takes compressed or poorly formatted CSS code and reformats it with proper indentation, line breaks, and spacing. This makes the CSS much easier to read, debug, and maintain. Minified CSS is great for production but unreadable for developers — a prettifier reverses this process so you can work with the code comfortably. ## What is CSS minification? CSS minification is the process of removing all unnecessary characters from CSS code without changing its functionality. This includes removing whitespace, line breaks, comments, and redundant semicolons. Minified CSS files are smaller in size, which means faster downloads and improved page load times. Most production websites serve minified CSS to reduce bandwidth and improve performance. ## How do I prettify CSS in my build process? You can use tools like **Prettier** , **stylelint** , or **csscomb** to automatically format CSS in your build pipeline. With Prettier, for example, you can format all CSS files by running: ```bash npx prettier --write "**/*.css" ``` You can also integrate these tools into your editor (VS Code, Sublime Text, etc.) for automatic formatting on save, or add them to pre-commit hooks using tools like **husky** and **lint-staged**. ## What is the difference between CSS prettify and CSS lint? CSS prettifying only reformats the appearance of your CSS (indentation, spacing, line breaks) without changing its meaning. CSS linting, on the other hand, analyzes your CSS for potential errors, bad practices, and style violations. A linter like **stylelint** can catch issues such as duplicate properties, invalid values, specificity problems, and vendor prefix issues. Ideally, you use both: lint to catch bugs and prettify to keep the code readable. ## Does prettifying CSS change how it works? No. Prettifying CSS only changes the whitespace and formatting of the code. The browser interprets prettified and minified CSS identically. The only difference is file size — prettified CSS is larger because of the added spaces, tabs, and newlines, which is why minified CSS is preferred for production. ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/test-effort-estimator [Tools](https://testingbot.com/free-online-tools)Calculators # Test Effort Estimator Plan your QA effort with precision Estimate total testing hours based on project parameters, complexity level, team size, and automation coverage. Get a detailed breakdown of manual testing, automation setup and regression cycles to plan your QA effort effectively. ## Project Parameters Number of Test Cases Complexity Level Low – Simple UI / Smoke tests Medium – Integration / API tests High – E2E / Cross-browser tests Critical – Security / Performance tests Team Size (QA Engineers) Automation Coverage: 30% 0% 100% Regression Cycles ## Effort Breakdown Initial Manual Testing 17.5 hrs Automation Setup 22.5 hrs Regression (Manual) 52.5 hrs Regression (Automated) 2.3 hrs * * * Total Effort 94.8 hrs Estimated Duration 6 working days 💡 Automation saves ~5 hours vs fully manual approach Manual-only would take 100 hrs total ## What is the Test Effort Estimator? The Test Effort Estimator is a free online tool that helps QA managers, test leads, and software testers estimate the total testing hours required for a project. By inputting project parameters such as the number of test cases, complexity level, team size, automation coverage, and planned regression cycles, you get a detailed breakdown of effort across different testing activities. This tool is especially useful during sprint planning, project scoping, and resource allocation to ensure realistic timelines and adequate QA coverage. ## How is the Test Effort Calculated? The estimator uses industry-standard formulas to calculate effort across four categories: - **Initial Manual Testing:** Based on the number of test cases multiplied by a base time-per-case (adjusted for complexity), covering the portion not automated. - **Automation Setup:** Time needed to write and configure automated tests, calculated from the automated test count with a higher per-case time investment. - **Regression (Manual):** Hours spent re-running manual tests across multiple regression cycles, factoring in the non-automated portion. - **Regression (Automated):** Machine execution time for automated regression runs, which is significantly faster than manual execution. The total effort is the sum of all four categories, and the estimated duration divides total hours by team size and an 8-hour working day. ## How Does Automation Coverage Affect Testing Effort? Automation coverage has a significant impact on total testing effort, particularly during regression cycles: - **Higher automation** reduces manual regression hours dramatically since automated tests run much faster. - **Initial investment** in automation setup is higher per test case, but pays off over multiple regression cycles. - **Break-even point:** For most projects, automation becomes cost-effective after 2-3 regression cycles. - **ROI insight:** The tool shows you how many hours automation saves compared to a fully manual approach, helping justify automation investments. ## How to Use the Test Effort Estimator ### 1. Enter Your Test Cases Input the total number of test cases in your test plan. This includes all functional, integration, and end-to-end test scenarios you plan to execute. ### 2. Select Complexity Level Choose the complexity that best matches your project: Low for simple UI/smoke tests, Medium for integration/API tests, High for E2E/cross-browser tests, or Critical for security/performance tests. ### 3. Set Team Size Enter the number of QA engineers available. This affects the estimated duration (working days), not the total effort in hours. ### 4. Adjust Automation Coverage Use the slider to set what percentage of your test cases will be automated. The tool dynamically recalculates all estimates as you adjust. ### 5. Set Regression Cycles Enter how many regression cycles you expect during the project. More cycles amplify the benefit of automation coverage. ## What Complexity Level Should I Choose? - **Low (Simple UI / Smoke tests):** Basic validation, simple form submissions, navigation checks. Typically 15 minutes per test case. - **Medium (Integration / API tests):** API endpoint testing, database interactions, multi-step workflows. Approximately 30 minutes per test case. - **High (E2E / Cross-browser tests):** Full end-to-end scenarios, cross-browser compatibility, multi-device testing. About 55 minutes per test case. - **Critical (Security / Performance tests):** Security audits, load testing, penetration testing, compliance checks. Around 75 minutes per test case. ## How Many Regression Cycles Should I Plan For? The number of regression cycles depends on your development methodology and release schedule: - **Agile sprints:** Typically 1-2 regression cycles per sprint - **Major releases:** 3-5 regression cycles before a major version release - **Continuous delivery:** May have many small regression runs, but with high automation coverage - **Waterfall projects:** Often 2-4 full regression cycles during the testing phase A good rule of thumb: if you expect more than 3 regression cycles, investing in automation coverage above 40% is strongly recommended. ## Can I Use This for Agile Sprint Planning? Yes, the Test Effort Estimator is ideal for agile sprint planning. Enter the number of test cases for the sprint, select the appropriate complexity, and set regression cycles to 1 or 2. The estimated duration helps you allocate QA capacity within the sprint and identify when you need additional resources or more automation coverage to meet sprint goals. ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/test-coverage-calculator [Tools](https://testingbot.com/free-online-tools)Calculators # Test Coverage Calculator Measure your testing completeness Calculate key QA metrics including requirements coverage, test execution rate, pass rate and defect density. Get a quality score with letter grade to quickly assess your project's test health. ## Input Metrics Total Requirements Covered Requirements Total Test Cases Executed Test Cases Passed Test Cases Total Code Lines Code Coverage: 72% 0% 100% Quality Score 82.3% Grade: B 81.3% Requirements Coverage 90.0% Execution Coverage 91.7% Pass Rate 1.50 Defect Density / KLOC Score = Req Coverage (30%) + Execution (20%) + Pass Rate (20%) + Code Coverage (30%) Defect Density = Failed Cases / (Code Lines ÷ 1000) ## What is the Test Coverage Calculator? The Test Coverage Calculator is a free online tool that helps QA teams measure how thoroughly their software is being tested. It calculates a weighted quality score based on four key metrics: requirements coverage, test execution rate, pass rate, and code coverage, then assigns a letter grade (A through F) for quick assessment. By tracking these metrics, teams can identify coverage gaps, prioritize testing efforts, and make data-driven decisions about release readiness. ## What Metrics Does the Calculator Provide? - **Requirements Coverage:** The percentage of requirements that have at least one test case mapped to them. Formula: (Covered Requirements / Total Requirements) x 100. - **Execution Coverage:** The percentage of total test cases that have been executed. Formula: (Executed Test Cases / Total Test Cases) x 100. - **Pass Rate:** The percentage of executed tests that passed successfully. Formula: (Passed Test Cases / Executed Test Cases) x 100. - **Defect Density:** The number of failed test cases per thousand lines of code (KLOC). Formula: Failed Cases / (Code Lines / 1000). Lower values indicate better code quality. - **Quality Score:** A weighted composite score combining requirements coverage (30%), execution coverage (20%), pass rate (20%), and code coverage (30%). ## How is the Quality Score Calculated? The quality score is a weighted average of four metrics: - **Requirements Coverage (30% weight):** Are all features covered by tests? - **Execution Coverage (20% weight):** Have all planned tests been run? - **Pass Rate (20% weight):** How many executed tests are passing? - **Code Coverage (30% weight):** What percentage of the codebase is exercised by tests? The letter grades are: A (90-100%), B (80-89%), C (70-79%), D (60-69%), F (below 60%). A score of B or above generally indicates good test health for most projects. ## How to Use the Test Coverage Calculator ### 1. Enter Requirements Data Input the total number of requirements and how many have test coverage. ### 2. Input Test Execution Data Enter total test cases, how many were executed, and how many passed. ### 3. Add Code Metrics Enter your total code lines (used for defect density calculation) and adjust the code coverage slider to match your project's coverage percentage. ### 4. Review Your Score All metrics and the quality score update in real-time. Use the letter grade and individual metric cards to identify areas for improvement. ## What is a Good Test Coverage Percentage? Industry benchmarks vary by project type, but general guidelines are: - **Requirements Coverage above 90%:** Excellent. Nearly all features have test coverage. - **Requirements Coverage 70-90%:** Good. Most critical features are covered, but gaps exist. - **Requirements Coverage below 70%:** Needs improvement. Significant risk of undiscovered defects. - **Code Coverage above 80%:** Strong coverage for most projects. - **Code Coverage 60-80%:** Acceptable, but critical paths should be verified. - **Pass Rate above 95%:** Indicates a stable, high-quality build. Keep in mind that 100% coverage does not guarantee a bug-free product. The quality of your test cases matters as much as the quantity. ## What is Defect Density and Why Does it Matter? Defect density measures the number of defects (failed test cases) relative to the size of the codebase, expressed per thousand lines of code (KLOC). It helps normalize defect counts across projects of different sizes. - **Below 1.0 / KLOC:** Good quality. The codebase has relatively few defects per unit of code. - **1.0-5.0 / KLOC:** Moderate. Common for projects in active development. - **Above 5.0 / KLOC:** High defect density. May indicate systemic code quality issues. Tracking defect density over time helps teams evaluate whether process improvements are reducing defect rates. ## Can I Use This for Sprint or Release Reporting? Yes, the Test Coverage Calculator is ideal for sprint retrospectives and release readiness reports. Input your sprint's test data to generate a quality score and coverage metrics that can be shared with stakeholders. The letter grade provides an instant, easy-to-understand summary, while the individual metrics offer the detail needed for planning improvements. ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) - [Test Plan Generator](https://testingbot.com/free-online-tools/test-plan-generator) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/test-plan-generator [Tools](https://testingbot.com/free-online-tools)Generators # Test Plan Generator Build comprehensive test plans in seconds Generate professional test plans with scope, approach, schedule and risk assessment. Fill in your project details and get a ready-to-use test plan document in Markdown format. ## Plan Details Project Name Objective In Scope Out of Scope Test Types Functional Integration E2E Performance Security Accessibility Regression Smoke UAT Start Date End Date Team Size Risks (comma-separated) ## Preview Copy ## What is the Test Plan Generator? The Test Plan Generator is a free online tool that helps QA managers and test leads create comprehensive, professional test plans in seconds. Simply fill in your project details including scope, test types, schedule, and risks, and the tool generates a structured Markdown document ready for sharing with your team. The generated plan follows industry-standard formats covering objectives, scope, test types, schedule, entry/exit criteria, risks and mitigations, and deliverables. ## What Does a Good Test Plan Include? A comprehensive test plan should include these key sections: - **Objective:** A clear statement of what the testing effort aims to achieve. - **Scope:** What features and areas are included (in scope) and excluded (out of scope). - **Test Types:** The types of testing to be performed (functional, integration, E2E, performance, etc.). - **Schedule:** Start and end dates, team size, and key milestones. - **Entry/Exit Criteria:** Conditions that must be met to start and conclude testing. - **Risks & Mitigations:** Identified risks and planned mitigation strategies. - **Deliverables:** Documents and reports to be produced during and after testing. ## How to Use the Test Plan Generator ### 1. Enter Project Details Provide your project name and a clear testing objective that describes what you aim to validate. ### 2. Define Scope Specify what is in scope (features to test) and out of scope (areas excluded from testing). ### 3. Select Test Types Check the types of testing you plan to perform. Common choices include Functional, Integration, E2E, and Regression. ### 4. Set Schedule and Team Enter your start date, end date, and team size. These appear in the schedule section of your plan. ### 5. Add Risks List project risks as comma-separated values. Each risk appears with a placeholder for mitigation strategies. ### 6. Copy and Share The preview updates in real-time. Click the Copy button to copy the Markdown to your clipboard, then paste it into your project wiki, Confluence, or documentation tool. ## What are Entry and Exit Criteria? **Entry criteria** define the conditions that must be met before testing can begin. Common examples include: requirements are signed off, test environment is available, and test data is prepared. **Exit criteria** define the conditions that signal testing is complete. Typical exit criteria include: all critical test cases executed, no open P1/P2 defects, and test coverage meets the target threshold. Clearly defining both prevents premature test starts and ensures a consistent quality bar before releases. ## What Test Types Should I Include? - **Functional Testing:** Validates that features work according to requirements. - **Integration Testing:** Tests interactions between modules, APIs, and services. - **E2E Testing:** Verifies complete user workflows from start to finish. - **Performance Testing:** Measures response times, throughput, and resource usage under load. - **Security Testing:** Identifies vulnerabilities, authentication issues, and data protection gaps. - **Accessibility Testing:** Ensures the application is usable by people with disabilities (WCAG compliance). - **Regression Testing:** Confirms that existing functionality still works after code changes. - **Smoke Testing:** Quick validation that critical features work before deeper testing. - **UAT (User Acceptance Testing):** End users validate the system meets their business needs. ## Can I Export the Generated Test Plan? Yes, the test plan is generated in Markdown format, which is widely supported across documentation tools. Click the Copy button to copy the full plan to your clipboard, then paste it into: - Confluence, Notion, or other wiki tools (most support Markdown paste) - GitHub/GitLab issues or wiki pages - Google Docs (paste and it will auto-format) - Jira or other project management tools - Any text editor or IDE with Markdown preview ## How Often Should a Test Plan Be Updated? A test plan should be treated as a living document and updated whenever there are significant changes to the project scope, timeline, or requirements. Key triggers for updates include: - New features added or requirements changed - Timeline shifts or resource changes - New risks identified during development - Lessons learned from previous test cycles For agile projects, consider regenerating or updating the plan at the start of each major sprint or release cycle. ### All Tools #### Accessibility - [Color Contrast Checker](https://testingbot.com/free-online-tools/color-contrast-checker) #### Calculator - [Test Effort Estimator](https://testingbot.com/free-online-tools/test-effort-estimator) - [Test Coverage Calculator](https://testingbot.com/free-online-tools/test-coverage-calculator) #### Converter - [Unix Timestamp Converter](https://testingbot.com/free-online-tools/unix-timestamp-converter) #### Formatter - [Puppeteer to Playwright Converter](https://testingbot.com/free-online-tools/puppeteer-to-playwright-converter) - [JSON Prettifier](https://testingbot.com/free-online-tools/json-prettify) - [HTML Escaper](https://testingbot.com/free-online-tools/html-escape) - [XML Prettifier](https://testingbot.com/free-online-tools/xml-prettify) - [Remove Whitespace](https://testingbot.com/free-online-tools/remove-all-whitespace) - [Text to One Line](https://testingbot.com/free-online-tools/text-to-one-line) - [PX to REM Converter](https://testingbot.com/free-online-tools/px-to-rem-converter) - [CSS Prettifier](https://testingbot.com/free-online-tools/css-prettify) #### Generator - [Random Avatar Generator](https://testingbot.com/free-online-tools/free-avatar-generator) - [Placeholder Image Generator](https://testingbot.com/free-online-tools/placeholder-image-generator) - [Random Address Generator](https://testingbot.com/free-online-tools/random-address-generator) - [Credit Card Generator](https://testingbot.com/free-online-tools/credit-card-number-generator) - [Random Number Generator](https://testingbot.com/free-online-tools/random-number-generator) - [JWT Parser](https://testingbot.com/free-online-tools/jwt-parser) - [Fake Person Generator](https://testingbot.com/free-online-tools/fake-person-generator) - [Hash Text](https://testingbot.com/free-online-tools/hash-text) - [Lorem Ipsum Generator](https://testingbot.com/free-online-tools/lorem-ipsum-generator) - [Random IP Generator](https://testingbot.com/free-online-tools/random-ip-generator) - [Random GUID Generator](https://testingbot.com/free-online-tools/random-guid-generator) - [Character Count](https://testingbot.com/free-online-tools/character-count) - [ASCII to Binary](https://testingbot.com/free-online-tools/ascii-to-binary) - [Random Color Generator](https://testingbot.com/free-online-tools/random-color-generator) - [Open Graph Meta Generator](https://testingbot.com/free-online-tools/og-meta-generator) - [QR Code Generator](https://testingbot.com/free-online-tools/qr-code-generator) - [Device Information](https://testingbot.com/free-online-tools/device-information) - [User Agent Parser](https://testingbot.com/free-online-tools/user-agent-parser) - [Image Extractor](https://testingbot.com/free-online-tools/image-extractor) #### Network - [CORS Tester](https://testingbot.com/free-online-tools/cors-tester) - [My GPS Location](https://testingbot.com/free-online-tools/my-gps-location) - [What's My IP Address](https://testingbot.com/free-online-tools/whats-my-ip-address) - [IP Subnet Calculator](https://testingbot.com/free-online-tools/ip-subnet-calculator) #### Scheduler - [Cron Expression Generator](https://testingbot.com/free-online-tools/cron-expression-generator) #### Speed - [Critical CSS Generator](https://testingbot.com/free-online-tools/critical-css-generator) #### Testing - [XPath & CSS Selector Tester](https://testingbot.com/free-online-tools/xpath-css-selector-tester) [View all tools](https://testingbot.com/free-online-tools) --- URL: https://testingbot.com/free-online-tools/xpath-css-selector-tester [Tools](https://testingbot.com/free-online-tools)Testing # XPath & CSS Selector Tester Test element selectors for Selenium, Cypress and Playwright Paste your HTML, write a selector and instantly see matched elements with ready-to-use code snippets for popular test automation frameworks. Supports CSS, XPath, ID, Name, Class Name, Tag Name and data-testid selectors. ## Input HTML Source \