You may not realize this, but Apache Spark is a framework. Spark is cluster-computing engine that manages parallel executions extremely well. Spark enables other technologies including Java, Scala, Python, R, and graph processing.

Spark stitches together previously-disparate functionality into a cohesive, syntactically-similar set of commands.

Spark’s architecture is library-driven and includes the following libraries:

Spark SQL
MLib (Machine Learning)
GraphX
Spark Streaming

When built correctly, frameworks are elegant solutions. What do I mean by elegant? For me, elegant is a description of the artistic side of the science and art components of software solutions. One attribute of an elegant solution is clever people will find additional uses for the tool, platform, or utility – applications the original developer never considered.

Spark is an elegant solution. It’s open source, and many developers are engaged in extending the platform ‘s functionality.

SSIS Framework

In similar fashion, the DILM (Data Integration Lifecycle Management) Suite‘s SSIS Framework supports enterprise data integration by providing a mechanism to knit together various SSIS packages into a single SSIS Application. Combined with SSIS Design Patterns and Biml (Business Intelligence Markup Language), an SSIS Framework is a powerful component of enterprise data integration DevOps with SSIS.

Most tools and utilities at DILM Suite are free and some are even open source.

For a limited time, you can purchase SSIS Catalog Compare, CatCompare (the Catalog Compare CLI), and SSIS Framework Commercial and Enterprise Editions at a 10% discount using the discount code 10PercentHoliday2017! Hurry, this offer expires 3 Jan 2018.

o<:{>