We performed a comparison between Apache Spark and Spark SQL based on real PeerSpot user reviews.
Find out in this report how the two Hadoop solutions compare in terms of features, pricing, service and support, easy of deployment, and ROI."It provides a scalable machine learning library."
"Its scalability and speed are very valuable. You can scale it a lot. It is a great technology for big data. It is definitely better than a lot of earlier warehouse or pipeline solutions, such as Informatica. Spark SQL is very compliant with normal SQL that we have been using over the years. This makes it easy to code in Spark. It is just like using normal SQL. You can use the APIs of Spark or you can directly write SQL code and run it. This is something that I feel is useful in Spark."
"We use Spark to process data from different data sources."
"Provides a lot of good documentation compared to other solutions."
"The product’s most valuable features are lazy evaluation and workload distribution."
"The solution is scalable."
"The data processing framework is good."
"I found the solution stable. We haven't had any problems with it."
"The stability was fine. It behaved as expected."
"Certain data sets that are very large are very difficult to process with Pandas and Python libraries. Spark SQL has helped us a lot with that."
"Offers a variety of methods to design queries and incorporates the regular SQL syntax within tasks."
"The speed of getting data."
"The solution is easy to understand if you have basic knowledge of SQL commands."
"The team members don't have to learn a new language and can implement complex tasks very easily using only SQL."
"It is a stable solution."
"One of Spark SQL's most beautiful features is running parallel queries to go through enormous data."
"It would be beneficial to enhance Spark's capabilities by incorporating models that utilize features not traditionally present in its framework."
"There were some problems related to the product's compatibility with a few Python libraries."
"When using Spark, users may need to write their own parallelization logic, which requires additional effort and expertise."
"Its UI can be better. Maintaining the history server is a little cumbersome, and it should be improved. I had issues while looking at the historical tags, which sometimes created problems. You have to separately create a history server and run it. Such things can be made easier. Instead of separately installing the history server, it can be made a part of the whole setup so that whenever you set it up, it becomes available."
"Apache Spark provides very good performance The tuning phase is still tricky."
"It should support more programming languages."
"They could improve the issues related to programming language for the platform."
"When you first start using this solution, it is common to run into memory errors when you are dealing with large amounts of data."
"In terms of improvement, the only thing that could be enhanced is the stability aspect of Spark SQL."
"This solution could be improved by adding monitoring and integration for the EMR."
"The solution needs to include graphing capabilities. Including financial charts would help improve everything overall."
"It takes a bit of time to get used to using this solution versus Pandas as it has a steep learning curve."
"I've experienced some incompatibilities when using the Delta Lake format."
"In the next release, maybe the visualization of some command-line features could be added."
"Anything to improve the GUI would be helpful."
"It would be beneficial for aggregate functions to include a code block or toolbox that explains its calculations or supported conditional statements."
Apache Spark is ranked 1st in Hadoop with 60 reviews while Spark SQL is ranked 4th in Hadoop with 14 reviews. Apache Spark is rated 8.4, while Spark SQL is rated 7.8. The top reviewer of Apache Spark writes "Reliable, able to expand, and handle large amounts of data well". On the other hand, the top reviewer of Spark SQL writes "Offers the flexibility to handle large-scale data processing". Apache Spark is most compared with Spring Boot, AWS Batch, SAP HANA, Cloudera Distribution for Hadoop and AWS Lambda, whereas Spark SQL is most compared with IBM Db2 Big SQL, SAP HANA, HPE Ezmeral Data Fabric and Netezza Analytics. See our Apache Spark vs. Spark SQL report.
See our list of best Hadoop vendors.
We monitor all Hadoop reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.
