ExtremeXP’s envisions to provide accurate, precise, fit-for-purpose, and trustworthy data-driven insights via evaluating different complex analytics variants, considering end users’ preferences and feedback in an automated way. ExtremeXP is motivated by significant gaps identified in current state of the art in big data analytics and automated machine learning, which lead to underutilizing user expertise, insights and feedback. Several existing big data frameworks and architectures provide support of processing, analytics, ML, simulations, and visualizations for large data volumes, given the proper infrastructure; however, they mainly focus on efficiency and scalability in pre-designed data analytic workflows. Existing Automated ML (AutoML) frameworks streamline the whole process of providing optimised ML models from data ingestion and pre-processing to model selection and training, to model execution and result visualization; however, they do not involve end (business) users in the loop.
To address such limitations, ExtremeXP proposes a new paradigm for data analytics, which we call experimentation-driven analytics. The main contribution is that it puts the end user, i.e. requirements, preferences, constraints, interpretation, explanations, feedback, and decision making, at the centre of complex analytics processes (from data discovery to novel interactions), proposing a human-in-the-loop experimentation approach for gaining knowledge and making decisions from data with varying and extreme characteristics.
An experiment for ExtremeXP considers alternative workflow variants (considering datasets, features, algorithms, models, simulations, visualizations) in order to respond to a user intent (expressed via preferences or constraints), executes them, and evaluates them based on both system-level metrics (latency, accuracy, precision, specificity, anonymity) and feedback from the user in an automated or semi-automated way. ExtremeXP integrates interactive visualization and explainability techniques to increase the trustworthiness of the outcomes as well as the process followed to reach such outcomes. To achieve the above, ExtremeXP aims to produce the following results:
• Modelling framework and reference architecture for complex experiment-driven analytics.
• Experimentation engine for automating the scheduling, evaluation, and adaptation of complex analytics.
• Analysis-aware data integration concept and methods.
• Methods for Automated ML (AutoML) with user constraints.
• Support for user involvement in complex experiment-driven analytics.
• Explainability-oriented user interaction toolset.
• Interactive visualisation support including augmented reality and serious games.
• Holistic data and knowledge management supporting privacy and security.
• Five successful pilot demonstrators to validate ExtremeXP through deployment in relevant environments.
