Cloudera Teams With NVIDIA to Cut Cloud Costs and Accelerate Spark Pipelines
New NVIDIA GPU acceleration for Apache Spark in Cloudera Data Engineering delivers zero-code performance gains, lower cloud costs, and faster AI-ready data pipelines
Cloudera,(opens in new tab) the only company bringing AI to data anywhere, today announced native GPU acceleration for Apache Spark 4.1 in Cloudera Data Engineering, enabled by the NVIDIA CUDA-X library, cuDF. The NVIDIA cuDF plug-in for Apache Spark will support the just-announced Cloudera Anywhere Cloud™, which is designed to enable organizations to accelerate Spark workloads without rewriting PySpark or SQL code. This helps data teams prepare AI-ready data faster while reducing cloud infrastructure costs across hybrid environments.
As organizations expand AI initiatives, the speed of data preparation has become a critical challenge. Large-scale Spark workloads often take hours to complete, delaying analytics and AI applications while driving up cloud compute costs. By embedding GPU acceleration directly into Cloudera Data Engineering, organizations can dramatically reduce processing times using their existing Spark applications, without changing code or operational workflows.
Also Read: IT Tech Pulse Exclusive Interview with Ken Claffey Chief Executive Officer and President of VDURA
Accelerating Spark for Enterprise AI
Apache Spark powers many of today’s enterprise data pipelines. With native GPU acceleration built into Cloudera Data Engineering, organizations can improve performance while maintaining the security and governance required for production workloads. Leveraging NVIDIA cuDF for Spark workloads, Cloudera will provide up to 4x workload acceleration on NVIDIA GPUs compared to traditional CPU infrastructure, enabling enterprises to drastically shorten data processing cycles.
Together, Cloudera Data Engineering accelerated by NVIDIA CUDA-X™ libraries will deliver:
- Zero-code GPU acceleration for Apache Spark 4.1 workloads
- Faster ETL and data preparation for analytics and AI
- Lower cloud infrastructure costs through shorter compute runtimes
- Built-in deployment with no manual driver configuration
- Enterprise security and governance through the Cloudera Unified Data Fabric
- Consistent performance across public cloud, private cloud, sovereign cloud, and on-premises environments
Unlike GPU acceleration offerings limited to a single cloud provider, Cloudera extends these capabilities across hybrid environments while preserving consistent governance and operations. Organizations can accelerate Spark workloads wherever their data resides— whether that is on premises, in the clouds, or at the edge— without sacrificing security or flexibility.
Faster Data Pipelines, Lower Infrastructure Costs
Preparing trusted data quickly, and at a reasonable cost, is essential for analytics and AI. Recent data from Cloudera’s The Great Re-Architecture Survey found that the vast majority (84%) of respondents said that AI workloads have caused infrastructure costs to increase. By accelerating Spark processing, organizations can deliver clean, model-ready data faster, helping to reduce infrastructure costs while improving productivity across data engineering, AI, and analytics teams.
Also Read: IT Tech Pulse Exclusive Interview with Pete Johnson Field CTO, Artificial Intelligence at MongoDB
“For many organizations, AI isn’t limited by models. It’s limited by how quickly they can turn raw data into trusted, usable insights,” said Leo Brunnick(opens in new tab), Chief Product Officer at Cloudera. “Accelerating Spark inside Cloudera Data Engineering helps remove that bottleneck, allowing customers to move from data preparation to analytics and AI faster while keeping governance, security, and operational consistency at the center of their strategy.”
“The fastest path to accelerating AI deployments is the one that aligns with how enterprises already operate today,” said Pat Lee,(opens in new tab) vice president, Strategic Enterprise Partnerships at NVIDIA.(opens in new tab) “With NVIDIA AI infrastructure and CUDA-X libraries now native to Cloudera Data Engineering, enterprises can lower costs and dramatically speed up Apache Spark pipelines without changing a single line of PySpark or SQL code, turning business data into a foundation for AI.”
The GPU acceleration capability for Apache Spark will be available in Cloudera Data Engineering as part of the Cloudera Anywhere CloudTM announced at EVOLVE Singapore on August 20, 2026. Additional demonstrations and technical sessions will be featured at NVIDIA GTC Berlin and Cloudera EVOLVE New York later this year.
Write to us [wasim.a@demandmediaagency.com] to learn more about our exclusive editorial packages and programmes.
