Intelligent Data Centres Issue 01 | Page 58

Pinakin Patel, Head of Solutions Engineering at MapR, explores the growing trend towards a new type of data middleware – aka ‘dataware’ – that is designed to do for data what middleware did for operating systems and applications. nformation Technology has an overriding progression centred on performance and efficiency. The mainframes of the 1970s moved to the racks of pizza boxes in data centres of the 1990s that are now the virtualised clusters within clouds. On the application side, the highly proprietary applications interfaces have given way to more open APIs such as SQL, RestAPI and S3. I One of the drivers is to remove complexity and an early example is the introduction of middleware as an abstraction layer. The Google dictionary description states of middleware: ‘Software that acts as a bridge between an operating system or database and applications, especially on a network.’ However, in the modern era, the stack of hardware, middleware and applications all connected by a network is much less certain. With smartphone apps, the cloud, shared networks, web-apps and a whole host of hybrid systems, the role of middleware as a glue to bind operating systems to applications is still valid, but middleware does not provide a consistent conduit for the handling of digital data which is the foundation for most business use cases. Data evolution In the past, where most data was generated from a big relational database, the integration was simple. Today, we have the data of a growing number of formats and delivery types. In no order, data can be structured and unstructured, file, object, streaming real-time, inference data and archived, plus several hybrid types. The data might need a certain type of encryption, higher availability or accessibility to a third party for governance issues. It might have regulatory needs that mean it can only be stored in a certain geography or for a certain duration. The list of data centric requirements along with a need for scaleable performance is vast, complex and growing. To address this issue, there is a growing trend towards a new type of data middleware – termed ‘dataware’ – that is designed to do for data what middleware did for operating systems and applications. Dataware sits in the spectrum between hardware and middleware, as a conceptual layer to create the next level of abstraction in the IT stack. Distinctly different from databases or data warehousing, dataware provides a platform-based approach that handles all data in terms of ingest, storage, availability, transformation and The growing trend towards ‘dataware’ 58 Issue 01