Apache JMeter vs Visual Studio Test Professional comparison

Cancel
You must select at least 2 products to compare!
Apache Logo
7,770 views|6,334 comparisons
87% willing to recommend
Microsoft Logo
862 views|720 comparisons
97% willing to recommend
Comparison Buyer's Guide
Executive Summary

We performed a comparison between Apache JMeter and Visual Studio Test Professional based on real PeerSpot user reviews.

Find out what your peers are saying about Apache, OpenText, Tricentis and others in Performance Testing Tools.
To learn more, read our detailed Performance Testing Tools Report (Updated: May 2024).
772,649 professionals have used our research since 2012.
Featured Review
Quotes From Members
We asked business professionals to review the solutions they use.
Here are some excerpts of what they said:
Pros
"We use Apache JMeter for load testing, where we provide the throughput time.""I like the fact that JMeter integrates well with other tools.""A lot of things are valuable. It is free. It has a lot of features, such as report generation and integration with CI/CD, which makes it very competitive with the other paid solutions available in the market. It is a good solution.""The reporting features are really good. There's a lot less latency than other solutions.""We appreciate that the solution is free to use, as an open-source tool.""We find the load testing feature valuable.""It's very easy to install, and it's very easy to code and develop the script.""JMeter is easy to use for a user who doesn't have too much knowledge of programming or certain languages."

More Apache JMeter Pros →

"The most valuable feature is the in-built support for C# and .NET projects.""It is a very common and strong product. A lot of support is available for this product.""The documentation is easy, and it helps us solve our problems.""The most valuable features are the SSIS reports, the deployment models, and the ability to interact with other Microsoft tools.""The ability to quickly make your own components has been valuable.""The solution is easy to use and they have also integrated with Microsoft.""The most valuable features of the solution are its ease of use and availability.""It is very easy to use. You can handle a lot of things together at once in one package, which is a good point for us."

More Visual Studio Test Professional Pros →

Cons
"What needs improvement in Apache JMeter is the very high load requirements when you want to scale it beyond certain thresholds. For example, small to mid-range testing is very easily done with Apache JMeter, but if you scale and increase the load, then it would be a problem because the tool consumes a lot of resources, probably because Apache JMeter provides an enriched UI experience, so it consumes a lot of memory and requires high CPU usage. This means you have to manage your infrastructure, or else you'll have high overhead expenses. As Apache JMeter is a heavyweight tool, that is an area for improvement, though I'm unsure if Apache can do something about it because it could be a result of the way it's architected. What I'd like to see from Apache JMeter in the future is for it to transition to the cloud, as a lot of cloud technologies emerge around the globe, and a lot of people prefer cloud-based solutions or cloud-native tools. Even if a company has a legacy system, it's still possible to transition to the cloud. I've worked with a company that was an on-premise company that moved to the cloud and became cloud-native. If Apache JMeter could transition to the cloud, similar to k6, then it could help lessen the intense resource consumption that's currently happening in Apache JMeter.""The solution's setup could be easier and security could be improved to minimize vulnerabilities.""The reporting is not very good.""Automation is difficult in JMeter.""It should be easier to combine multiple scripts. If you have multiple scripts, you need to write a new script to combine those scripts. The virtual user generator is slow.""In this tool, automation in general is almost non-existent. Everything is done manually.""Improving JMeter's sync time would be beneficial.""At times when we overload the application, it gets stuck...After the solution gets stuck due to overloading, we have to restart our computers. In short, the solution keeps crashing."

More Apache JMeter Cons →

"One of the problems with this solution is you need to be highly technically skilled to operate it, it is not for everyone.""The integration with Git needs improving because it is a bit disjointed and unpredictable.""Visual Studio Test Professional is a little pricey.""The server that we use is very slow so that is concerning for us.""The price could be improved.""The documents on the Microsoft website are not very useful, and they ought to make it easier to find answers.""Sometimes, the solution hangs, so its performance could be improved.""The solution is quite expensive."

More Visual Studio Test Professional Cons →

Pricing and Cost Advice
  • "Free"
  • "I switched to Apache because it is free. Other tools are much too expensive and can cost up to $50,000 a year if you are looking at commercial options."
  • "This is an open-source solution, and there are no fees."
  • "This is an open-source product."
  • "I was using the free version of the software."
  • "In terms of open-source adoption, it is completely free."
  • "We are using the free version, and if required, we can easily switch to the other version."
  • "When comparing the price with Load Runner, and if the cost is an issue then JMeter is a better choice"
  • More Apache JMeter Pricing and Cost Advice →

  • "For the cloud services option, you buy a subscription per account or per user. This costs around $52 a month per person."
  • "I think that the pricing is quite good."
  • "The pricing is expensive."
  • "We pay for the solution annually and the price could be reduced."
  • "There is a paid version of the solution as well as a community version that is free."
  • "Visual Studio Test Professional is a very expensive solution."
  • "The tool is expensive in my region."
  • "We pay a yearly licensing fee for Visual Studio Test Professional, which is expensive."
  • More Visual Studio Test Professional Pricing and Cost Advice →

    report
    Use our free recommendation engine to learn which Performance Testing Tools solutions are best for your needs.
    772,649 professionals have used our research since 2012.
    Questions from the Community
    Top Answer:Postman lets you easily define variables, which then get updated automatically. This is a huge time-saver and makes processes very efficient. We can also export the test cases we create and share them… more »
    Top Answer:Blazemeter is a continuous testing platform that provides scriptless test automation. It unifies functional and performance testing, enabling users to monitor and test public and private APIs We… more »
    Top Answer: I appreciate JMeter's simplicity and power for performance testing.
    Top Answer:The most valuable features of the solution are its ease of use and availability.
    Top Answer:Visual Studio Test Professional is not an expensive solution.
    Top Answer:The solution's documentation could be improved because it keeps disappearing from the solution. There used to be references material that were incorporated in the solution, but most of it has moved to… more »
    Ranking
    Views
    7,770
    Comparisons
    6,334
    Reviews
    43
    Average Words per Review
    480
    Rating
    7.7
    7th
    Views
    862
    Comparisons
    720
    Reviews
    30
    Average Words per Review
    278
    Rating
    8.6
    Comparisons
    Also Known As
    JMeter
    Learn More
    Overview

    Apache JMeter is an open-source Java application that tests load and functional behavior and performance in applications. Created initially to test web applications, it has expanded its functionality to test other functions. For instance, you can test a server to see how efficiently it works and how many user requests can be handled simultaneously.

    You can use JMeter to test functional performance and regression tests on different technologies. This Java desktop application has an easy-to-use graphical interface which uses the Swing graphical API. You can run JMeter on any environment that accepts a Java virtual machine, such as Windows, Linux, and Mac.

    What protocols does JMeter support?

    • Web - HTTP, HTTPS
    • Web services - SOAP/XML-RPC
    • Email service - POP3, IMAP, SMTP
    • FTP service
    • Database via JDBC drivers
    • LDAP
    • Native commands or shell scripts
    • TCP
    • Java objects

    How does JMeter work?

    JMeter sends requests to a target server by simulating a group of user requests. Then it collects and calculates statistics on the performance of the target. This target can be a server or an application.

    You can test the performance of static resources, such as JavaScript or HTML, and dynamic resources, such as JSP, Servlets, and AJAX. It is also helpful to determine how many concurrent users your website can handle.

    There are two main tests you can carry out with JMeter: load test and stress test. The load test models expected usage of a server by simulating multiple users accessing the web server simultaneously. The stress testing aims to find the maximum load capacity of the server or application.

    Apache JMeter Key Features

    • JMeter enables fast test plan recording, building, and debugging via a featured test IDE (integrated development environment).
    • Command-line mode allows carrying out load tests from any OS compatible with Java.
    • JMeter can extract data from most popular response formats, such as HTML, JSON, XML.
    • JMeter is entirely portable.
    • A multithreading framework allows you to simultaneously test multiple samplings and separate thread groups.
    • Thanks to the caching feature, you can conduct offline analysis or replay test results.

    Apache JMeter Benefits

    • As open-source software, it is freely available and continuously improved.
    • The setup is user-friendly, and no installation is needed.
    • The GUI (graphic user interface) is intuitive and easy to use.
    • You can write your own tests and use the visualization plugins to extend the testing.
    • It is a platform-agnostic tool. For example, JMeter can work with Linux by clicking on the JMeter shell script. On Windows, you can call up JMeter by starting the jmeter.bat file.
    • Since the test plans are stored in XML format, you can generate a test plan using any text editor.
    • You can simulate a heavy load on a server, a group of servers, a network, or an object. Use JMeter to test an application or server's strength and performance under different load types.

    The JMeter extensible core has numerous benefits:

    • Unlimited testing capabilities via pluggable samplers.
    • You can choose multiple load statistics with pluggable timers.
    • Visualization plugins and data analysis enable customization.
    • Continuous integration via third-party open-source libraries (Maven, Gradle, and Jenkins).

    Reviews from Real Users

    Stephen B., I.T. Architect, Analyst, and Developer at an educational organization, says, "The scripting ability is most valuable. It is easy to use. There is a UI, and you can go in there and figure those things out. After you've got a good set of tests, you basically have a scripted document that you can grab and execute in a pipeline. It is pretty quick to set up, and you can scale it and version control it."

    "I like the fact that JMeter integrates well with other tools," adds the Founder and Principal Consultant at a tech services company.

    A Quality Engineering Delivery Leader at a financial services firm says, “The performance of the solution is excellent. They have designed the product so that it is very easy to configure. You can basically do anything you like with the product. It's not very restrictive. We like the fact that the technology is open-source.”

    Visual Studio Professional Edition provides an IDE for all supported development languages. As of Visual Studio 2010, the Standard edition was dropped. MSDN support is available as MSDN Essentials or the full MSDN library depending on licensing. It supports XML and XSLT editing, and can create deployment packages that only use ClickOnce and MSI. It includes tools like Server Explorer and integration with Microsoft SQL Server also. Windows Mobile development support was included in Visual Studio 2005 Standard, however, with Visual Studio 2008, it is only available in Professional and higher editions. Windows Phone 7 development support was added to all editions in Visual Studio 2010. Development for Windows Mobile is no longer supported in Visual Studio 2010; it is superseded by Windows Phone 7.
    Sample Customers
    AOL, Orbitz, Innopath Software, PrepMe, Sapient, Corporate Express Australia, CSIRO, Ephibian, Talis, DATACOM, ALALOOP, eFusion, Panter, Sourcepole, University of Western Cape
    Transport for Greater Manchester, Ordina, Bluegarden A/S, CLEAResult, Jet.com, OSIsoft, Australian Taxation Office, BookedOut, Tracasa
    Top Industries
    REVIEWERS
    Computer Software Company38%
    Financial Services Firm30%
    Energy/Utilities Company4%
    Renewables & Environment Company4%
    VISITORS READING REVIEWS
    Financial Services Firm18%
    Computer Software Company17%
    Manufacturing Company8%
    Government6%
    REVIEWERS
    Computer Software Company38%
    Financial Services Firm9%
    Healthcare Company6%
    Manufacturing Company6%
    VISITORS READING REVIEWS
    Computer Software Company18%
    Financial Services Firm16%
    Manufacturing Company8%
    Insurance Company7%
    Company Size
    REVIEWERS
    Small Business25%
    Midsize Enterprise24%
    Large Enterprise52%
    VISITORS READING REVIEWS
    Small Business16%
    Midsize Enterprise13%
    Large Enterprise70%
    REVIEWERS
    Small Business24%
    Midsize Enterprise28%
    Large Enterprise48%
    VISITORS READING REVIEWS
    Small Business19%
    Midsize Enterprise14%
    Large Enterprise67%
    Buyer's Guide
    Performance Testing Tools
    May 2024
    Find out what your peers are saying about Apache, OpenText, Tricentis and others in Performance Testing Tools. Updated: May 2024.
    772,649 professionals have used our research since 2012.

    Apache JMeter is ranked 1st in Performance Testing Tools with 82 reviews while Visual Studio Test Professional is ranked 7th in Functional Testing Tools with 48 reviews. Apache JMeter is rated 7.8, while Visual Studio Test Professional is rated 8.6. The top reviewer of Apache JMeter writes "It's a free tool with a vast knowledge base, but the reporting is lackluster, and it has a steep learning curve". On the other hand, the top reviewer of Visual Studio Test Professional writes "Customization is a key feature as is the ability to integrate with third-party services ". Apache JMeter is most compared with BlazeMeter, Postman, Tricentis NeoLoad, OpenText LoadRunner Professional and OpenText Silk Performer, whereas Visual Studio Test Professional is most compared with TFS, Tricentis NeoLoad, SmartBear TestComplete, Tricentis Tosca and TestRail.

    We monitor all Performance Testing Tools reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.