We performed a comparison between SmartBear TestComplete and TestRail based on real PeerSpot user reviews.
Find out in this report how the two Test Automation Tools solutions compare in terms of features, pricing, service and support, easy of deployment, and ROI."The most valuable feature is the integration with Azure DevOps."
"The solution is mainly stable."
"The product is stable for what we are currently using it for, and it is sufficient for us."
"The reporting is ready to use and doesn't require any setup."
"The ability to run a whole suite of tests automatically (which we did overnight)."
"It is a strong automation tool for desktop, browser, and API testing."
"It works very fine. It is fast on almost any machine, and it is also very well organized. I like its object mapping and its capability to find and interact with almost everything that exists on Windows."
"Complete works perfectly with CUTE. That includes all dialogues, right-click menus, or system dialogues, etc., which are handled well."
"You don't need to follow complex procedures to create a test run, test case, etc."
"Most valuable features are the ease of organizing test cases and a great API for sending results from automated test run results into the database."
"The most valuable features of TestRail by Gurock are the user experience, it's very easy to learn. There is no learning curve needed to work on projects and manage the test cases, it is easy. Exporting and importing are simple."
"The features that I have found most valuable are that there are various test case templates and test artifact maintenance."
"The API to support integration of the homemade automated testing tool."
"The product’s most valuable feature is the UI. The structure of test cases is easy to understand."
"I use the solution for test management."
"Reliable and stable. It is important that TestRail be up and running 24/7 as we have users around the world using it."
"Stability issues occurred only when connecting to the SourceSafe. Sometimes, after getting the latest version, the tool hangs and it should be reopened in order to recover."
"Product is not stable enough and it crashes often."
"Name Mapping feature should be clearer. Whenever I use it, I do not really know what will work and what will not work."
"Error handling features in the tool are a little limited."
"The solution needs more training manuals or some form of online forum for learning. It needs more documentation."
"At times, identifying or locating an element can be somewhat challenging. However, in a recent test update, they introduced Optical Character Recognition (OCR) capability. This introduction has reduced the challenges to some extent, as we can now utilize OCR if the normal method doesn't work. Nevertheless, there is still significant potential for improvement in TestComplete's ability to identify various object elements. I don't have any specific concerns to mention. I have observed significant improvements in TestComplete over the past few years, especially in its support for highly dynamic object elements used in products like Salesforce Dynamics 365. In earlier versions, there were numerous challenges, but the current version is far superior to its predecessors."
"During the distribution of our regression test cases, the control IDs are not always recognized correctly."
"The integration tools could be better."
"With TestRail, the APIs are there, but they may not be able to easily integrate with the Jenkins."
"I do see room for lots of improvement in it. In terms of usability, duplication with test cases and constant creation of projects isn't easy. There is also too much API integration into automation tools, which is not there in ALM with UFT. Instead of setting it up as a project and using it, we set it up as a system for usability. It also lacks in the traceability aspect. For traceability, you need to use the JIRA plugin and drag traceability on JIRA, but the functionality is still quite limited. The biggest gap is mainframe testing. It would be good if I could start with mainframe testing. Manual granting of access is another issue. There is no API that I could use with another system where it is automated. There is an API for loading somebody to a project but not for adding to the application."
"I've encountered at some point, some difficulties on the administration side, but I don't remember exactly what they were."
"There are a number of improvements that have been requested. While I don't have a list of these requests available, many can be found on Gurock's forum."
"I have faced some issues with the integration between TestRail and Jira, which haven't been permanently resolved yet."
"The platform needs improvement regarding performance and creating links."
"It's not easy to create a custom report. It's not straightforward. A good improvement would be if there was a way to report and create a custom report without using a plugin or scripting language."
"The test suite management has room for improvement as well as better reporting."
SmartBear TestComplete is ranked 7th in Test Automation Tools with 71 reviews while TestRail is ranked 2nd in Test Management Tools with 21 reviews. SmartBear TestComplete is rated 7.6, while TestRail is rated 8.0. The top reviewer of SmartBear TestComplete writes "A stable product that needs to improve its integration capabilities with other test management tools". On the other hand, the top reviewer of TestRail writes "A tool that provides effective test management and real-time reporting capabilities". SmartBear TestComplete is most compared with Tricentis Tosca, Katalon Studio, Ranorex Studio, OpenText UFT One and froglogic Squish, whereas TestRail is most compared with Zephyr Enterprise, Tricentis qTest, TFS, Tricentis Tosca and Visual Studio Test Professional. See our SmartBear TestComplete vs. TestRail report.
We monitor all Test Automation Tools reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.